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ABSTRACT 


Currently  Department  of  Defense  (DoD)  policy  requires  programs  to  develop 
architectural  products  as  part  of  programmatic  documentation.  Specifically,  the  Joint 
Capabilities  Integration  and  Development  System  (JCIDS)  and  DoD  5000  series  requires 
architecture  products  at  acquisition  milestone  decisions.  The  DoD  implements  a 
recommended  framework,  the  Department  of  Defense  Architecture  Framework 
(DoDAF),  which  describes  these  architectures. 

The  purpose  of  this  project,  suggested  by  Air  Force  Space  Command,  was  to  examine 
the  value  of  existing  analytical  tools  in  making  an  interoperability  assessment  of 
individual  enterprises,  as  well  as  assess  the  touch-points  between  enterprise  architectures. 
This  novel  evaluation  scheme  is  based  solely  on  the  architecture  products,  rather  than  the 
more  common  assessment  via  interviews  of  subject  matter  experts  or  actual  system 
testing.  If  the  architecture  products  required  by  DoD  are  to  have  any  merit,  their 
underlining  data  must  be  used  by  decision  makers.  Well  developed  architectures  can 
better  aid  in  capability  planning,  investment  decisions  (i.e.  spiral  upgrades),  as  well  as 
support  proposals  for  integrated  Family  of  Systems  solutions  by  identifying  gaps. 

The  project  examines  the  application  of  two  different  assessment  tools  to  three 
different  enterprise  architectures;  these  included  the  DoD’s  Global  Information  Grid 
(GIG),  the  Air  Force  C2  Constellation  (C2C)  and  the  Combatant  Commanders  Integrated 
Command  and  Control  System  (CCIC2S).  Lastly,  some  suggested  recommendations  for 
improving  both  the  architectural  products  and  tools  to  aid  in  interoperability  assessments. 
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1  Introduction 

1. 1  Background  and  Project  History 

Initially  this  effort  began  on  a  much  different  path,  investigating  the  possibility  of 

developing  an  initial  architecture  supporting  responsive  space,  sponsored  by  the  Air 
Force  Research  Lab  (AFRL).  During  the  research  of  the  problem  it  was  discovered  that 
Air  Force  Space  Command  (AFSPC)  had  several  initiatives  underway,  with  several 
products  soon  to  be  delivered.  Additionally,  a  meeting  with  representatives  from  the 
National  Security  Space  Office  (NSSO)  provided  focus  towards  the  development  of  a 
responsive  space  architecture  at  the  national  level.  Following  this  meeting  with  the 
NSSO,  the  possibility  of  the  capstone  project  adding  value  to  either  AFRL  or  the  NSSO 
was  the  first  shift  in  direction. 

It  was  during  the  investigation  of  responsive  space  where  the  idea  of  investigating 
enterprise  architectures  to  support  the  NSSO  strategic  effort  came  to  light.  Following  a 
meeting  with  AFSPC/DRN,  the  investigation  of  Air  Force  Space  Command  enterprise 
architectures  began  to  take  shape.  AFSPC  posed  several  questions,  one  of  which  was, 
“Explain  what  it  means  to  integrate  enterprise  architectures,  and  define  test  criteria  for 
judging  the  quality  or  degree  of  architecture  integration.”1  As  the  literature  search  and 
brainstonning  continued,  to  the  drive  was  for  conducting  an  interoperability  examination 
of  several  architectures.  The  problem  was  scoped  upon  the  idea  of  comparing  the 
interoperability  of  architectures  using  two  existing  architecture  evaluation  tools,  each 
developed  for  different  purposes. 


1 


1.2  Purpose 

The  purpose  of  this  project  is  to  use  three  enterprise  architectures  to  examine  existing 
tools  to  make  an  assessment  on  the  potential  for  characterizing  interoperability  across 
architectures.  In  doing  this,  the  ground  rules  are  to  only  utilize  existing  architecture 
products  and  determine  if  they  contained  sufficient  information  to  make  an  assessment. 
This  approach  is  counter  to  the  more  common  technique  of  conducting  extensive  surveys 
or  interviews  of  subject  matter  experts  (SME).  SMEs  may  have  their  own  biases  which 
often  times  differ  from  the  published  documentation  of  the  system. 

1.3  Research  Objective 

In  light  of  DoD  5000.1  requirement  for  interoperability  of  infonnation  systems,  the 
focus  is  on  examination  of  three  architectures  for  interoperability  from  an  enterprise 
perspective.  The  three  architectures  are  the  Combatant  Commanders  Integrated 
Command  and  Control  System  (CCIC2S)  from  Air  Force  Space  Command,  the 
Command  and  Control  Constellation  (C2C)  from  Electronic  Systems  Center,  and  the 
Global  Infonnation  Grid  (GIG)  from  the  Department  of  Defense.  The  two  existing  tools, 
Levels  of  Information  Systems  Interoperability’s  InspeQtor  and  the  Enterprise 
Architecture  Score  Card  were  created  to  examine  individual  architecture,  but  both  discuss 
the  ability  to  characterize  system  interoperability.  While  conducting  the  literature 
review,  it  appeared  that  most  assessments  of  architectures  or  systems  were  conducted 
through  some  type  of  interview.  The  project  goes  the  other  way  in  looking  at 
architectures  as  stand-alone  products,  because  the  SMEs  may  not  always  be  working  with 
the  system.  The  objective  was  to  assess  the  interoperability  of  the  architectures  through 
the  use  of  existing  analytical  tools  for  making  a  judgment  on  the  interoperability  of 
architectures.  The  desired  outcome  is  identifying  possible  tools  which  help  in  the 
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decision  process  by  making  an  interoperability  assessment  across  systems.  Nevertheless, 
due  to  the  subjective  nature  of  this  assessment,  some  actual  systems  interoperability 
testing  may  be  required  to  validate  these  recommendations." 

2  Literature  Review 

2.1  Overview 

This  section  covers  an  examination  of  DoD  policy,  some  definitions  key  to  the 
remainder  of  the  project,  a  review  of  the  architecture  framework  and  an  overview  of  the 
three  architectures  being  examined.  A  synopsis  of  the  two  tools  utilized  in  this  project  is 
covered  in  the  next  section  of  this  paper. 

2.2  Policy 

Recently  systems  engineering  and  architectures  have  received  a  renewed  emphasis 
within  the  Air  Force.  One  indication  of  this  comes  from  Dr  James  Roche,  Secretary  of 
the  Air  Force  in  a  24  June  2002  Air  Force  Times  article.  In  response  to  questions  dealing 
with  recent  acquisition  program  problems  he  stated  “Increasingly,  I’m  convinced  that  the 
systemic  problem  is  in  the  field  of  systems  engineering.”  In  fact,  senior  Air  Force 
Leadership  recently  adjusted  developmental  education  for  its  officers  to  provide  an 
increased  focus  on  systems  engineering. 

Moving  up  a  level  in  the  DoD  hierarchy,  architectures  are  an  important  element  of  the 
relatively  new  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  process. 
This  is  the  joint  process  for  identifying  and  resolving  gaps  in  military  capability.  In  terms 
of  formal  policy,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  3 170.01D, 
Joint  Capabilities  Integration  and  Development  System  Instruction  and  CJCSI  62 12.0 1 , 
Interoperability  and  Supportability  of  National  Security  Systems,  and  Information 
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Technology  Systems  directly  impact  program  managers  and  engineers  in  program  offices. 
CJCSI  3170  establishes  the  policies  and  procedures  for  “a  joint  concepts-centric 
capabilities  identification  process.”4  The  JCIDS  calls  for  specific  DoD  Architecture 
Framework  (DoDAF)  architecture  views  as  annexes  to  documents  such  as  the  Initial 
Capability  Document  (ICD),  Capability  Development  Document  (CDD),  and  Capability 
Production  Document  (CPD)  to  support  acquisition  milestone  decisions  on  moving 
forward  with  system  development  programs.  Additionally,  CJCSI  6212  “details  a 
methodology  to  develop  interoperability  Key  Performance  Parameters... based  on  the 
format  and  content  of  the  integrated  architecture  products  described  in  the  most  current 
version  of  the  DoDAF.”5 

Beyond  the  JCIDS  requirements,  programs  are  required  to  produce  architectures  as 
part  of  the  required  documentation  for  Information  Support  Plans  (ISPs  -  formerly  C4I 
Support  Plans)  and  as  part  of  the  documentation  required  to  identify  net-ready  key 
performance  parameters  (NR-KPP).6  Figure  1  identifies  the  architecture  views  required 
for  support  of  various  documents  in  the  JCIDS  process  and  the  use  in  deriving  the  NR- 
KPP.  Of  note  is  the  column  on  the  far  right  of  the  table  requiring  systems  to  have 
different  levels  of  completeness  in  the  Levels  of  Information  System  Interoperability 
(LISI)  model,  which  is  one  of  the  tools  discussed  later  to  measure  interoperability. 
According  to  Chairman  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  6212. 01C,  “ah  CDDs 
(Capability  Development  Documents)  that  exchange  information  will  have  a  NR-KPP” 
which  is  “derived  from  a  completed  architecture  and  developed  from”  mandatory 
architecture  products.  In  fact,  the  instruction  goes  on  to  say  “development  of  the  NR- 
KPP  begins  with  designing  the  architecture  for  the  proposed  system.”7 
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Figure  1:  JCIDS  Doeuments/NR-KPP  Products  Matrix  (CJCSI,  2003) 

All  of  these  development  programs  are  budgeted,  programmed  and  financed  through 
the  DoD’s  Planning,  Programming,  Budgeting  and  Execution  (PPBE)  System. 

Originally,  PPBE  was  introduced  in  the  1960’s  by  then  Secretary  of  Defense  Robert 
McNamara  to  create  DoD  budgets  that  better  achieve  Department  objectives  via 
integration  of  the  infonnation  necessary  to  craft  effective  plans  and  programs.  This 
system  is  the  primary  DoD  resource  management  system  to  articulate  strategy;  identify 
force  size,  structure,  and  equipment;  set  programming  priorities;  allocate  resources  for 
operations,  acquisition,  and  other  functions  within  fiscal  constraints;  and  evaluate  actual 
outputs  against  planned  performance,  adjusting  resources  as  appropriate.  It  was  modified 
slightly  by  Secretary  Rumsfeld  in  2003  to  reflect  current  budgeting  practices.  The  PPBE 
reflects  the  current  status  of  today’s  programs,  in  essence  if  the  program  does  not  have 
funds  allocated  in  PPBE  it  does  not  exist. 
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The  DoD  followed  this  rationale  in  conducting  a  Net-Centric  Program  Assessment  in 
the  summer  of  2004.  The  stated  purpose  of  this  assessment  was  “to  help  components  and 
programs  with  the  Net-Centric  Operations  Warfare  Reference  Model,  and  to  establish 
priorities  and  design  transition  plans  for  embracing  the  spirit  and  intent  of  Global 
Information  Grid  (GIG)  Enterprise  Services.”8  The  results  of  the  assessment  were  then  to 
be  used  by  the  DoD  Chief  Information  Officer  (CIO)  to  assist  in  departmental 
deliberation  of  resource  allocation  input  into  the  President’s  Budget  for  Fiscal  Years 
2006-2011. 

Coming  back  to  the  service  level,  the  Air  Force’s  rationale  for  insisting  on  improved 
systems  engineering  comes  from  the  increasing  complexity,  sophistication,  and  cost  of 
current  weapon  systems  programs,  an  issue  for  systems  engineering  help  to  guide  the 
program  through  development.  Howard  Eisner  offers  the  following  in  his  book, 
Essentials  of  Project  and  Systems  Engineering  Management,  “Architecting  a  large-scale 
complex  system  is  the  centerpiece  of  systems  engineering.”9  The  AFIT  course  on 
Systems  Architectures  presented  the  view  that  architectures  are  necessary  when  doing 
new  and  complex  things.  For  example,  one  would  not  hire  an  architect  to  build  a  house 
in  a  development,  but  would  if  they  were  branching  out  to  build  something  that  had  not 
been  done  before.  Following  this  analogy,  the  system  architecture  serves  as  a  blueprint 
for  the  program  and  within  the  acquisition  process,  “systems  architecting  is  an  essential 
part  of  the  system  engineering  process  and  relies  on  many  of  the  methodologies  that  have 
been  developed  over  time.”10 
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2.3  Definitions 

2.3.1  Enterprise  Architectures 

Basically  enterprise  architecture  explains  the  business  processes,  relationships, 
guidance  and  policy  that  characterize  the  environment  in  which  an  organization  exists. 
Maier  and  Rechtin,  in  their  work  the  Art  of  System  Architecting,  say  an  enterprise 
specification  of  an  Open  Distributive  Processing  system  is  a  model  of  the  system  and  the 
environment  with  which  the  system  interacts.  It  covers  the  role  of  the  system  in  the 
business,  and  the  human  user  roles  and  business  policies  related  to  the  system.  The 
enterprise  viewpoint  is  a  viewpoint  on  the  system  and  its  environment  that  focuses  on  the 
purpose,  scope,  and  policies  of  the  system. 1 1 

Two  other  definitions  for  ‘enterprise’  during  are  pre-told.  The  first  comes  from  John 
Zachman’s  Concept  for  Framework  for  Enterprise  Architecture,  Background, 

Description  and  Utility.  Zachman’s  work  states  that  set  of  descriptive  representations 
(i.e.  ‘models’)  that  are  relevant  for  describing  an  enterprise  such  that  it  can  be  produced 
to  management’s  requirements  (quality)  and  maintained  over  the  period  of  its  useful  life 
(change).  The  second  view  comes  from  the  Federal  CIO  Council  which  claims  enterprise 
architecture  describes  the  "target"  situation  that  the  organization  wishes  to  create  and 
maintain. 12 

Perhaps  the  benefits  of  Enterprise  Architectures  as  explained  in  an  article  by  David 
Brown  in  spring  2000  will  help  put  things  in  context.  The  benefit  of  understanding  an 
organization’s  enterprise  architecture  is  to  create  a  more  efficient  organization;  not  just  a 
bunch  of  disparate,  individually  efficient  functions,  which  are,  organizationally 

1  T 

dysfunctional. 
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The  bottom  line  is  an  Enterprise  Architecture  is  different  from  simple  system 
architecture  in  that  it  does  not  focus  specifically  on  a  single  system  and  has  a  broader 
scope,  an  overarching  vision  and  concern  for  the  management  and  business  practices  of 
the  organization.  While  a  simple  architecture  may  capture  the  development  of  a  system 
or  lead  to  requirements  definition  for  a  new  acquisition,  the  Enterprise  Architecture 
incorporates  the  external  systems  environment  of  the  organization,  fielded  systems  and 
development  activities.  Like  a  simple  architecture,  an  Enterprise  Architecture  can 
decompose  into  a  system  of  systems  or  family  of  systems. 

2.3.2  Integration  Points 

Integration  points  are  defined  as  the  interfaces  between  two  systems  at  the  external 
systems  level.  Applying  this  to  the  systems  engineering  model  from  Buede’s  text, 
Engineering  Design  of  Systems  Models  and  Methods,  leads  to  the  picture  in  Figure  2. 
Integration  points  can  evolve  from  information  flows  (control,  input  and/or  output 
signals)  and  possibly  the  physical  interactions  (mechanisms)  between  systems.  It  is 
important  that  integration  points  must  be  commonly  defined  across  separate  architectures, 
their  scale  and  deconstruction  may  vary  by  architecture.  For  example,  portfolio 
management  architectures  need  not  include  as  many  views  as  one  used  for  technology 
investment  decisions.  An  operations  planning  and  execution  architecture  should  include 
all  seven  operational  views  but  may  only  need  one  or  two  systems  views,  compared  to  a 
systems  design  and  development  architecture  which  needs  more  systems  views  than 
operational  views.14  However,  integration  points  cannot  contradict  each  other.  Finally, 
integration  points  provide  a  common  core  for  joint  program  payoff  in  JCIDS  products  by 
ensuring  interoperability  and  reuse. 
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Figure  2:  Integration  Points 

2.3.3  Interoperability 

Interoperability  takes  on  many  different  meanings  to  different  people.  For  the 
purposes  of  this  project,  interoperability  is  the  connectivity  of  two  systems  to  flow 
information  freely  from  one  to  another  and  back  again. 

In  terms  of  formal  policy,  Chairman  of  the  Joint  Chiefs  of  Staff  Manual  (CJCSM) 

3 170.01a,  Operation  of  the  Joint  Capabilities  Integration  and  Development  System 
defines  interoperability  in  relation  to  the  Net-Ready  Key  Performance  Parameters  (KPP) 
as  “the  ability  of  systems,  units  or  forces  to  provide  data,  information,  materiel  and 
services  to  and  accept  the  same  from  other  systems,  units  or  forces  and  to  use  the  data, 
information,  materiel  and  services  so  exchanged  to  enable  them  to  operate  effectively 
together.  IT  and  NSS  interoperability  includes  both  the  technical  exchange  of 
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information  and  the  end-to-end  operational  effectiveness  of  that  exchanged  infonnation 
as  required  for  mission  accomplishment.”15 

2.4  Description  of  the  DoDAF 

A  better  understanding  of  what  architecture  is  can  be  demonstrated  by  a  discussion  of 
frameworks.  As  previously  mentioned,  these  architecture  products  are  part  of  a  certain 
framework,  such  as  the  Federal  Enterprise  Architecture  Framework,  The  Open  Group 
Architecture  Framework,  the  IEEE  1471  Standard,  the  Zachman  Framework,  or  the  DoD 
Architecture  Framework  (DoDAF).  Each  of  these  frameworks  is  slightly  different  with 
somewhat  different  products  recommended  by  each.  The  key  framework  for  the  analysis 
is  the  DoDAF  developed  “in  the  early  1990s  [as]  an  architecture  framework  for 
Command,  Control,  Communications,  Computing,  Intelligence,  Surveillance,  and 
Reconnaissance  (C4ISR)  systems.”16  A  short  discussion  on  the  DoDAF  is  applicable, 
particularly  for  the  reason  that  this  framework  is  most  applicable  to  Air  Force  System 
Engineers.  Additionally,  all  three  architectures  used  in  this  capstone  project  utilize  the 
DoDAF. 

Captain  Millette  outlined  the  views  of  the  DoDAF  in  his  thesis  with  the  following 
discussion. 17  The  DoDAF  consists  of  multiple  products  known  as  views.  There  are  four 
types  of  views,  the  All  Views,  Operational  Views,  Systems  Views,  and  Technical 
Standards  Views.  Several  of  these  views  are  collected  in  what  is  called  an  “integrated 
architecture”  referred  to  extensively  in  the  JCIDS  documentation.  These  views  are 
summarized  in  Table  1,  and  are  the  architecture  products  referred  throughout  this 
research  effort. 
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DoDAF  Volume  Two  defines  architecture  products  as:  Those  graphical,  textual, 

and  tabular  items  that  are  developed  in  the  course  of  gathering  architecture  data, 

identifying  their  composition  into  related  architecture  components  or  composites,  and 

modeling  the  relationships  among  those  composites  to  describe  characteristics  pertinent 

1 8 

to  the  architecture’s  intended  use. 

Thus,  architecture  products  can  take  the  fonn  of  Power  Point  charts,  Excel 
spreadsheets,  tables,  and  charts,  as  well  as  any  other  graphical  product  that  conforms  to 
the  standard  above.  The  DoDAF  is  careful  not  to  specify  a  certain  development 
methodology.  In  fact,  it  is  purposely  intended  to  be  methodology  independent. 

The  All  Views  category  captures  essential  overview  information  about  the 
architecture.  The  Overview  and  Summary  (AV-1)  is  essential  for  documenting  the 
assumptions,  constraints,  and  limitations  that  may  affect  high-level  decision  processes 
involving  architecture.  AV-1  also  identifies  the  approving  authority,  the  completion  date, 
and  records  level  of  effort  and  costs  required  to  develop  the  architecture  as  well  as  the 
time  frame  covered  and  the  organizations  that  fall  within  the  scope  of  the  architecture.19 

“The  Operational  View  (OV)  describes  the  tasks  and  activities  necessary  to 
successfully  perform  the  mission,  the  participating  nodes,  and  the  associated  infonnation 
exchanges.”  Further,  “OV  descriptions  are  useful  for... defining  the  operational 
requirements  to  be  supported  by  resources  and  systems”  and  “a  pure  OV  is  materiel 
independent.”'  In  order  to  deliver  a  weapon  system,  the  tasks  and  activities  modeled  in 
the  OVs  are  allocated  to  systems,  which  are  themselves  modeled  in  Systems  Views. 

“The  Systems  View  (SV)  describes  the  systems  of  concern  and  the  connections  among 
those  systems  in  context  with  the  OV.”'  Finally,  “the  Technical  Standards  View  (TV) 
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describes  a  profile  of  the  minimum  set  of  time-phased  standards  and  rules  governing  the 
implementation,  arrangement,  interaction,  and  interdependence  of  systems.”  "  The 
DoDAF  defines  an  integrated  architecture  (a  term  used  throughout  JCIDS  and  other 
documents)  as  the  AV-1,  AV-2,  OV-2,  OV-3,  OV-5,  SV-1,  and  TV-1,  at  a  minimum.23 


Table  1:  The  DoDAF  Views 


Applicable 

View 

Framework 

Product 

Framework  Product  Name 

General  Description 

All  Views 

AV-1 

Overview  and  Summary 
Information 

Scope,  purpose,  intended  users,  environment  depicted,  analytical  findings 

All  Views 

AV-2 

Integrated  Dictionary 

Architecture  data  repository  with  definitions  of  all  terms  used  in  all  products 

Operational 

OV-1 

High-Level  Operational 

Concept  Graphic 

High-level  graphical/textual  description  of  operational  concept 

Operational 

OV-2 

Operational  Node 

Connectivity  Description 

Operational  nodes,  connectivity,  and  information  exchange  needlines  between  nodes 

Operational 

OV-3 

Operational  Exchange  Matrix 

Information  exchange  between  nodes  and  the  relevant  attributes  of  that  exchange 

Operational 

OV-4 

Organizational  Relationships 
Chart 

Organizational,  role,  and  other  relationships  among  organizations 

Operational 

OV-5 

Operational  Activity  Model 

Capabilities,  operational  activities,  relationships  among  activities,  inputs,  and 
outputs;  overlays  can  show  cost,  performing  nodes,  or  other  pertinent  information 

Operational 

OV-6a 

Operational  Rules  Model 

One  of  three  products  used  to  describe  operational  activity  -  identifies  business  rules 
that  constrain  information 

Operational 

OV-6b 

Operational  State  Transition 
Description 

One  of  three  products  used  to  describe  operational  activity  -identifies  business 
processes  and  responses  to  events 

Operational 

OV-6c 

Operational  Event-Trace 
Description 

One  of  three  products  used  to  describe  operational  activity  -  traces  actions  in  a 
scenario  or  sequence  of  events 

Operational 

OV-7 

Logical  Data  Model 

Documentation  of  the  system  data  requirements  and  structural  business  process  rules 
of  the  operational  View 

Systems 

SV-1 

Systems  Interface  Description 

Identification  of  systems  nodes,  systems,  and  system  items  and  their 
interconnections  with  and  between  nodes 

Systems 

SV-2 

Systems  Communications 
Description 

Systems  nodes,  systems  and  system  items  and  their  related  communications  lay- 
downs 

Systems 

SV-3 

Systems- Systems  Matrix 

Relationships  among  systems  in  a  given  architecture;  can  be  designed  to  show 
relationships  of  interest,  e.g.,  system-type  interfaces,  planned  vs.  existing,  etc. 

Systems 

SV-4 

Systems  Functionality 
Description 

Functions  performed  by  systems  and  the  system  data  flows  among  system  functions 

Systems 

SV-5 

Operational  Activity  to 

Systems  Function 

Traceability  Matrix 

Mapping  of  systems  back  to  capabilities  or  system  functions  back  to  operational 
nodes 

Systems 

SV-6 

Systems  Data  Exchange 

Matrix 

Provides  details  of  system  data  elements  being  exchanged  between  systems  and  the 
attributes  of  that  exchange 

Systems 

SV-7 

Systems  Performance 

Parameter  Matrix 

Performance  characteristics  of  Systems  View  elements  for  the  appropriate  time 
frame(s) 

Systems 

SV-8 

Systems  Evolution 

Description 

Planned  incremental  steps  toward  migrating  a  suite  of  systems  to  a  more  efficient 
suite,  or  toward  evolving  a  current  system  to  a  future  implementation 

Systems 

SV-9 

Systems  Technology  Forecast 

Emerging  technologies  and  software/hardware  products  that  are  expected  to  be 
available  in  a  given  set  of  time  frames  and  that  will  affect  future  development  of  the 
architecture 

Systems 

SV-lOa 

Systems  Rules  Model 

One  of  three  products  used  to  describe  system  functionality  -  identifies  constraints 
that  are  imposed  on  systems  functionality  due  to  some  aspect  of  systems  design  or 
implementation 

Systems 

SV-1  Ob 

System  State  Transition 
Description 

One  of  three  products  used  to  describe  system  functionality  -  identifies  responses  of 
a  system  to  events 

Systems 

SV-lOc 

System  Event-Trace 

Description 

One  of  three  products  used  to  describe  system  functionality  -  identifies  system- 
specific  refinements  of  critical  sequences  of  events  described  in  the  Operational 

View 

Systems 

SV-11 

Physical  Schema 

Physical  implementation  of  the  Logical  Data  Model  entities 

Technical 

TV-1 

Technical  Standards  Profile 

Listing  of  Standards  that  apply  to  System  View  elements  in  a  given  architecture 

Technical 

TV-2 

Technical  Standards  Forecast 

Description  of  emerging  standards  and  potential  impact  on  current  Systems  View 
element,  within  a  set  of  time  frames 

Other  definitions  deal  with  architecture’s  role  in  the  design  of  the  system.  Howard 
Eisner,  author  of  Essentials  of  Project  and  Systems  Engineering  Management,  believes 
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architecting  is  “fundamentally  a  design  or  synthesis  process”  and  goes  on  to  define 

24 

architecture  as  “the  evolution  of  the  DoDAF.” 

The  DoD  broadened  the  application  of  the  framework  beyond  C4ISR  systems 
based  on  the  utility  of  the  C4ISR  Architecture  Framework  combined  with  both  Federal 
(Clinger-Cohen  Act  of  1996,  etc.)  and  DoD  policy  encouraging  the  use  of  architectures 
during  program  acquisition."  The  result  was  the  publication  in  2004  of  the  DoDAF 
Version  1.0  Volumes  I  and  II.  The  stated  purpose  of  the  DoDAF  Version  1.0,  is  “to 

provide  guidance  for  describing  architectures  for  both  warfighting  operations  and 
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business  operations  and  processes.”" 

2. 5  The  Enterprise  Architectures 

The  focus  is  on  examination  of  three  architectures  for  interoperability  from  an 

enterprise  perspective.  The  three  architectures  are  the  Combatant  Commanders 
Integrated  Command  and  Control  System  (CCIC2S)  from  Air  Force  Space  Command, 
the  Command  and  Control  Constellation  (C2C)  from  Electronic  Systems  Center,  and  the 
Global  Infonnation  Grid  (GIG)  from  the  Department  of  Defense. 

2.5.1  Command  and  Control  Constellation  (C2C) 

The  Command  and  Control  Constellation  is  the  U.S.  Air  Force's  force  packaging 

approach  to  optimize  the  Air  Component  and  maintaining  C2  and  Intelligence, 
Surveillance  and  Reconnaissance  (ISR)  capabilities  in  support  of  Joint  Operating 
Concepts.27  The  Electronic  Systems  Command,  along  with  the  AFC2ICRC  and  AF/XI, 
created  the  C2C  Architecture  to  support  C2C  definition  and  conceptual  design;  support 
requirements  development,  early  milestone  decision  reviews  and  concept  of  operation 
maturation;  facilitate  Global  Information  Grid  architecture  compliance;  prototype  uses  of 
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architecture  as  a  tool  to  align  capital  investments  as  required  by  the  Clinger-Cohen  Act 
and  facilitate  portfolio  management  at  both  the  micro-  and  macro-level. 

The  C2C  Architecture  documents  include  both  a  2005  “As-Is”  Architecture  and  a 
2012  “To-Be”  Architecture.  The  views  provided  are  listed  in  Table  2.  Some  views  are 
common  to  both  architectures  (the  TV-2  Technical  Standards  Forecast,  AV-1  Overview 
and  Summary  Information,  etc.)  MITRE  Corporation  created  architecture  views  using  a 
Structured  Analysis  approach.  Most  views  were  created  in  Popkin’s  System  Architect 
software  and  exported  to  Power  Point  briefing  slides,  while  other  products  were  created 
with  Microsoft  Excel. 


Table  2:  C2C  Architecture  Views  Provided 


AV-1 

Overview  and  Summary  Information 

OV-5 

2012  Node  Tree 

AV-2 

Integrated  Dictionary 

OV-6a 

2005  Operational  Rules  Model 

OV-1 

2005  High  Level  Operational  Concept 
Graphic 

OV-6a 

2012  Operational  Rules  Model 

OV-1 

2012  High  Level  Operational  Concept 
Graphic 

SV-4 

2005  Systems  Functionality  Description 

OV-2 

2005  Operational  Node  Connectivity 
Description 

SV-4 

2012  Systems  Functionality  Description 

OV-2 

2012  Operational  Node  Connectivity 
Description 

SV-5 

2005  Operational  Activity  to  Systems 
Function  Traceability  Matrix 

OV-3 

2005  Operational  Information  Exchange 
Matrix 

SV-5 

2012  Operational  Activity  to  Systems 
Function  Traceability  Matrix 

OV-3 

2012  Operational  Information  Exchange 
Matrix 

SV-9 

Systems  Technology  Forecast 

OV-5 

2005  Operational  Activity  Model 

TV-1 

Technical  Standards  Profile 

OV-5 

2012  Operational  Activity  Model 

TV-2 

Technical  Standards  Forecast 

OV-5 

2005  Node  Tree 

2.5.2  Combatant  Commanders  Integrated  Command  and  Control  System 
(CCIC2S) 

The  Combatant  Commanders  Integrated  Command  and  Control  System  is  designed  to 
support  Air  Force  Space  Command  (AFSPC),  North  American  Aerospace  Defense 
Command  (NORAD),  and  U.S.  Strategic  Command  (USSTRATCOM)  with  an 
integrated,  interoperable,  flexible,  and  cost  effective  capability  enabling  warfighters  to 
accomplish  their  current  and  evolving  missions.  The  purpose  of  the  CCIC2S  Operational 
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Architecture  is  to  capture  user  and  operator  requirements  for  detailed  design  and  test  of 
the  system.  This  will  enable  the  Commanders  of  NORAD,  USSTRATCOM  and  AFSPC 
to  carry  out  assigned  missions  contained  in  the  Unified  Command  Plan.  The  operational 
architecture  is  the  methodology  to  integrate  common  functions  across  34  separate 
operational  systems  and  27  different  programming  languages. 

The  CCIC2S  architecture  was  written  as  a  “To-Be”  architecture  for  an  unspecified 
date  of  implementation.  The  architecture  included  a  comprehensive  Operational 
Requirements  Document  linked  to  a  Hypertext  Markup  Language  (HTML)-based  viewer. 
Architectural  views  and  products,  listed  in  Table  3,  were  created  in  Rational  Rose 
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software  with  the  Unified  Modeling  Language  following  an  Object-Oriented  approach." 


Table  3:  CCIC2S  Architecture  Views  Provided 


AV-1 

Overview  and  Summary  Information 

OV-6a 

Use  Case  Specifications 

AV-2 

Integrated  Dictionary 

OV-6c 

Operational  Trace  Sequence  Diagrams 

OV-1 

High  Level  Operational  Concept  Graphic 

OV-7 

Logical  Data  Model 

OV-2 

Operational  Node  Connectivity  Description 

SV-5 

2005  Operational  Activity  Matrix 

OV-3 

Operational  Information  Exchange 
Requirement  (IER)  Matrix 

Functional  Performance  Requirements 
Mapping 

OV-4 

Operational  Command  Relationships 

Technical  Architecture 

OV-5 

Activity  Diagram 

Systems  Operational  Sequence  Verb  List 

OV-5 

Use  Case  Relationships  Diagrams 

Operational  Requirements  Document 
(ORD) 

OV-5 

Use  Case  Diagrams 

2.5.3  Global  Information  Grid  (GIG) 

The  Global  Information  Grid  provides  the  means  for  warfighters,  decision  makers, 

and  policymakers  to  conduct  and  support  military  operations.  The  GIG  is  a  physical 
entity  -  the  sum  of  the  Department’s  information  capabilities,  systems,  services,  and 
facilities,  and  associated  processes  and  personnel.29  The  GIG  Version  2  (v2) 
Architecture  was  designed  to  implement  Net-Centric  Operations  and  Warfare  (NCOW) 
for  future  conflicts.  It  serves  as  the  initial  architectural  description  of  NCOW  concepts, 
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terminology,  and  modeling.  It  supports  the  DoD  Chief  Information  Officer’s  decisions 
and  recommendations  concerning  Information  Technology  planning  programming 
acquisition  and  policy.  Finally,  it  identifies  the  enterprise  requirements  for  the  GIG  in  a 
net-centric  environment  with  all  programs  plugging  into  or  interfacing  with  the  GIG. 

The  GIG  v2  Architecture  applied  to  three  separate  levels  of  combat  operation  models 
in  Strategic,  Operational,  and  Tactical  Use  Cases.  The  Architecture  Views  are  listed  in 
Table  4.  All  three  use  cases  were  tightly  coupled  together  in  a  HTML-based  viewer. 
Views  were  created  with  Popkin's  System  Architect  software  using  a  Structured  Analysis 
approach. 


Table  4:  GIG  Architecture  Views  Provided 


Strategic  Use  Case 

Operational  Use  Case 

Tactical  Use  Case 

Secretary  of  Defense  Force 
Allocation 

Homeland  Defense 

Southwest  Asia  Warfighting 

AV-1 

Overview  and  Summary 
Information 

AV-1 

Overview  and  Summary 
Information 

AV-1 

Overview  and  Summary 
Information 

AV-2 

Integrated  Dictionary 

AV-2 

Integrated  Dictionary 

AV-2 

Integrated  Dictionary 

OV-1 

High  Level  Operational 
Concept  Graphic 

OV-1 

High  Level  Operational 
Concept  Graphic 

OV-1 

High  Level  Operational 
Concept  Graphic 

OV-2 

Operational  Node 
Connectivity 

OV-2 

Operational  Node 
Connectivity 

OV-2 

Operational  Node 
Connectivity 

OV-3 

Operational  Information 
Exchange  Requirements 

OV-3 

Operational  Information 
Exchange  Requirements 

OV-3 

Operational  Information 
Exchange  Requirements 

OV-4 

Command  Relationships 

OV-4 

Command  Relationships 

OV-4 

Command  Relationships 

OV-5 

Activity  Model 

OV-5 

Activity  Model 

OV-5 

Activity  Model 

OV-5 

Node  Tree 

OV-5 

Node  Tree 

OV-5 

Node  Tree 

SV-1 

Systems  Interface 

SV-1 

Systems  Interface 

SV-1 

Systems  Interface 

SV-2 

Systems  Communications 

SV-2 

Systems  Communications 

SV-2 

Systems  Communications 

SV-3 

System  Matrix 

SV-3 

System  Matrix 

SV-3 

System  Matrix 

SV-4 

System  to  System 

Functions 

SV-4 

System  to  System 

Functions 

SV-4 

System  to  System 

Functions 

SV-5 

System  Function  to 
Operational  Activity 

SV-5 

System  Function  to 
Operational  Activity 

SV-5 

System  Function  to 
Operational  Activity 

TV-2 

Technical  Standards  Forecast  document  is  common  for  all  three  Use  Cases  j 
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3  Tools 

Two  existing  tools,  Levels  of  Information  Systems  Interoperability’s  InspeQtor  and 
the  Enterprise  Architecture  Score  Card  were  created  to  examine  individual  architecture, 
but  both  discuss  the  ability  to  characterize  system  interoperability. 

3.1  The  Levels  of  Information  System  Interoperability  (LISI) 

Joint  Vision  2020,  the  vision  statement  for  the  DoD,  says  “interoperability  is  the 

foundation  of  effective  joint,  multinational,  and  interagency  operations.  The  joint  force 
has  made  significant  progress  toward  achieving  an  optimum  level  of  interoperability,  but 
there  must  be  a  concerted  effort  toward  continued  improvement.  Interoperability  is  a 
mandate  for  the  joint  force  of  2020  -  especially  in  terms  of  communications,  common 
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logistics  items,  and  information  sharing.” 

In  response  to  the  vision  of  the  Joint  Chiefs  as  laid  out  in  Joint  Vision  2020,  the 
C4ISR  Architecture  Working  Group  (AWG)  delivered  the  Levels  of  Information  System 
Interoperability  (LISI)  construct  in  1998.  The  purpose  of  the  LISI  construct  is  to  provide 
a  process  for  determining  interoperability  needs,  assessing  the  ability  of  specific 
information  systems  to  meet  those  needs,  and  selecting  pragmatic  solutions  and  transition 
paths  for  achieving  higher  states  of  capability  and  interoperability.  There  are  several 

key  concepts  in  this  construct’s  approach.  The  first  is  providing  an  interoperability 
maturity  model  to  describe  levels  of  sophistication  regarding  information  exchanges. 
Next,  LISI  provides  requirement  organizations  the  ability  to  identify  operational  and 
system  requirements  in  terms  of  interoperability.  Third,  the  LISI  construct  has  a  suite  of 
capabilities  associated  with  the  procedures,  applications,  infrastructure,  and  data  domains 
in  order  to  obtain  the  desired  level  of  capability.  Finally,  LISI  provides  a  practical 
assessment  process  for  determining  the  interoperability  of  a  given  system  or  across  a 
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system  pair.  This  process  uncovers  capabilities  that  may  be  lacking  in  the  systems,  areas 
not  compatible  and  options  for  resolving  the  deficiencies  so  the  systems  can  move  to  a 

T9 

higher  level  of  interoperability. 


3.1.1  LISI  Interoperability  Maturity  Model 
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Figure  3:  The  LISI  Interoperability  Maturity  Model 

The  LISI  construct  uses  five  increasing  levels  of  sophistication  regarding  system 
interaction  and  the  ability  of  the  system  to  exchange  and  share  information  and  services. 
These  levels  are  the  basis  of  the  Interoperability  Maturity  Model  as  shown  in  Figure  3. 
Each  higher  level  represents  a  demonstrable  increase  in  capabilities  over  the  previous 
level  of  system-to-system  interaction.  For  example,  a  system  that  shares  files  via  an  e- 
mail  attachment  would  operate  are  a  Level  1,  or  the  connected  state.  That  is  to  say  the 
ability  exists  to  transfer  files  electronically  between  or  with-in  the  systems,  but  it  requires 
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another  program  to  make  the  transfer.  On  the  other  end  of  the  spectrum  a  system  with  a 
common  database  that  shares  data  amongst  systems  components  would  be  at  the  Level  3 
or  Domain  level.  As  it  is  apparent  from  this  example,  the  higher  the  sophistication  of  the 
system,  the  higher  the  level  rating  in  the  LISI  Interoperability  Maturity  Model  and  the 
model  implies  that  a  higher  rating  is  the  desired  result.  With-in  these  maturity  levels, 
there  are  many  factors  which  LISI  groups  into  the  four  key  attributes  discussed  in  the 
next  section. 


3.1.2  LISI  Attributes 

This  construct  uses  four  areas  of  cohesion  to  organize  the  aspects  of  information 
systems.  The  four  attributes  are  Procedures,  Applications,  Infrastructure,  and  Data  that 
fit  together  like  a  puzzle,  as  shown  in  Figure  4,  to  describe  the  construct  while  providing 
the  unique  perspectives  of  purpose  and  identity. 


What  policies  and 
procedures  enable 
systems  to  exchange 
information, 
capabilities,  and 
services? 


What  set  of 
applications  enable 
information 
exchange,  processing, 
or  manipulation? 


Applications 


Infra¬ 
structure 


What  information 
formats,  data 
protocols,  or 
databases  enable  the 
exchange  of  data  and 
information? 


What  environment 
(hardware, 
communications  and 
networks,  system 
services,  and 
security)  enables 
system  interaction? 


Figure  4:  The  PAID  Attributes  Puzzle 

The  procedures  (P)  attribute  encompasses  the  many  forms  of  documented  guidance 
and  operational  controls,  standards  and  architecture  guidance,  which  affect  all  aspects  of 
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system  development,  integration,  and  operational  functionality.34  The  general  purpose  of 
the  procedures  attribute  is  to  address  implementation  of  the  system’s  interoperability. 

The  four  categories  in  procedures  attribute  are  as  follows: 

•  Standards  -  Compliance  with  existing  technical  standards,  architectures  and 
common  operating  environments 

•  Management  -  Covers  the  realm  of  program  management  from  system 
requirements  definition  to  delivery  and  turn-over  of  the  system 

•  Security  Policy  -  Covers  the  procedures  to  ensure  that  proper  security  is 
maintained  while  accessing  and  sharing  information. 

•  Operations  -  considers  the  operational  plans  of  the  system,  but  not  how  the 
system  operates.  For  example,  if  the  system  has  plans  for  e-mail  file  transfer  but 
the  mail  server  is  not  connected,  then  credit  is  given  because  the  system  is 
planned  to  use  this  method. 

The  purpose  of  the  applications  (A)  attribute  is  to  cover  the  fundamental  purpose  and 
function  for  which  any  system  is  built  —  its  mission.  Specifically  this  attribute  examines 
the  functional  requirements  specified  by  users  to  perform  an  operational  activity  and  how 
they  are  handled  by  the  software  application.  For  interoperability  to  occur  effectively 
similar  capabilities  or  a  common  understanding  of  the  shared  information  must  exist 
between  systems;  otherwise,  users  have  no  common  frame  of  reference. 

The  third  attribute,  Infrastructure  (7),  supports  the  establishment  and  use  of  a 
“connection”  between  systems  or  applications.  It  does  not  matter  if  the  connection  is 
handled  on  a  simple  sneaker-net  approach  where  files  are  transferred  via  magnetic  media 
and  walked  from  component  to  component,  or  transferred  via  complex  integrated 
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wireless  networks.  There  is  also  a  security  aspect  involved  in  this  attribute,  specifically 
the  devices  and  technical  capabilities  that  are  used  to  implement  the  procedures  attribute. 
Lastly,  the  attribute  also  includes  the  core  system  services  such  as  the  communication 
protocols  that  support  and  govern  operations  and  interactions. 

The  final  attribute,  data  (D),  focuses  on  the  information  or  data  processed  by  the 
system.  This  deals  with  both  the  data  format  and  the  content  of  the  information.  Data 
covers  a  wide  range  of  style  and  format,  but  generally  it  is  typed  as  either  homogeneous 
or  heterogeneous.  Homogeneous  data  files  are  composed  of  a  single  content  type  like  a 
text  file.  On  the  other  hand,  heterogeneous  data  files  consist  of  multiple  forms  of 
information  in  a  single  file  such  as  a  multimedia  document  or  annotated  image.  The 
Architecture  Working  Group  points  out  that  the  data  attribute  is  understandably  the  most 
critical  aspect  of  attaining  systems  interoperability.  It  is  within  this  attribute  where  much 
of  today’s  focus  and  work  towards  building  interoperable  systems  is  taking  place.  One 
example  of  this  work  is  the  use  of  the  extensible  Markup  Language  (XML),  an 
application-agnostic  language  used  for  exchanging  information.  Due  to  the  scope  and 
time  constraints,  this  area  is  recommended  for  a  future  AFIT  Capstone  Project. 

3.1.3  LISI  Reference  Model 

When  all  four  attributes  are  measured  using  the  LISI  Interoperability  Maturity  Model 
the  result  is  a  LISI  reference  model,  shown  in  Figure  5,  forms  the  basis  for  the  assessment 
of  a  given  system.  Systems  are  graded  based  upon  the  level  obtained  in  each  of  the  PAID 
categories.  The  overall  rating  for  the  system  is  determined  as  the  lowest  of  the  four 
category  ratings.  For  example,  if  a  system  is  rated  a  level  three  for  procedures, 
applications  and  data  but  a  level  1  for  infrastructure  the  overall  score  is  a  1 .  Each  level  is 


21 


further  broken  down  into  three  or  four  sub-levels  to  distinguish  advancement  of 
capabilities. 


Figure  5:  LISI  Reference  Model 

3.1.4  LISI  and  the  DoDAF 

In  developing  the  LISI  model,  MITRE  specifically  related  it  to  the  DoDAF  as  shown 
in  Figure  6.  Specifically  the  model  ties  to  the  Operational  Views  by  concentrating  on  the 
details  of  the  Information  Exchange  relationships.  It  provides  a  discipline  and 
methodology  for  discussing  and  documenting  these  relationships  along  with  a  metric  for 
characterization.  LISI  expands  this  by  providing  information  about  the  set  of  capabilities 
the  Systems  Views  use  to  answer  the  operational  requirements.  Lastly,  LISI  provides  an 
important  construct  and  a  bridge  to  the  prevailing  formal  technical  guidance  as  the  link  to 
the  Technical  Views  of  the  DoDAF.35 
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Figure  6:  LISI  Relationship  to  the  DoDAF  Views 

3.1.5  LISI  Shortcomings 

During  the  research  effort  a  few  critics  of  LISI  were  discovered,  who  brought  out 
valid  points  that  should  be  considered  before  using  this  model.  The  first  is  a  report  from 
Thea  Clark,  of  the  Defence  Science  and  Technology  Organisation  in  Canberra,  Australia 
titled  Organisational  Interoperability  Maturity  Model  for  C2.  The  premise  for  her 
arguments  is  that  the  LISI  construct  is  centered  strongly  on  technology,  and  more 
specifically,  as  its  name  suggests  on  information  exchanges.  The  construct  does  not 
address  the  higher  layers  of  C2  support  and  the  system-oriented  definitions  of 
interoperability  levels  do  not  seem  to  have  a  natural  extension  into  the  higher  layers  of 
the  model.  The  report  goes  on  to  discuss  possible  expansion  of  the  LISI  model  in 
consideration  of  interoperability.  Of  interest  is  the  distinction  the  report  makes  between 
interoperability  and  compatibility: 
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If  interoperability  is  defined  as  the  ability  of  one  entity  to  service  another 
then  compatibility  is  defined  as  the  degree  to  which  one  electronic  system 
can  operate  with  another  -  it  is  a  subset  of  interoperability.  Thus,  when 
looking  at  the  layers  of  C2  support,  compatibility  is  more  applicable  to  the 
lower  technological  layers  and  interoperability  to  the  higher  organisational 
layers. 

Where  interoperability  has  been  driven  by  process,  the  focus  is  on  the 
situation,  the  people  and  commander's  intent.  This  may  lead  to  flexible 
interoperability  but  not  necessarily  to  technical  compatibility:  This  may  be 
expressed  in  a  logical  format  as: 

Process  =>Flexible  Interoperability  =>  limited  technology  compatibility 

Where  interoperability  has  been  driven  by  technology,  the  focus  is  on 
assets,  their  properties  and  the  levels  of  compatibility  required.  This  may 
lead  to  the  exclusion  of  non-compatible  participants.  In  a  logical  format: 

Technology  =>  Planned  Interoperability  =>  limited  inter-working 


This  distinction  adds  credibility  to  the  approach  of  assessing  interoperability  of  an 
enterprise  level. 

Another  complaint  of  the  LISI  construct  is  in  its  difficulty  and  complexity  in 
implementation.  In  attempting  to  find  a  simple  measure  to  assess  interoperability  for 
Joint  Forces,  Dr.  Hamilton  from  Auburn  University  along  with  Major  Paul  Summers  and 
Captain  Jerome  Rosen  developed  this  opinion  of  LISI:  “At  its  core,  LISI,  is  based  around 
classifying  levels  of  interoperability  by  the  ‘richness’  of  communication  that  a  particular 
system  or  group  of  systems  allow.  We  believe  the  model  is,  at  root,  too  complicated  for 
use  in  aggregating  the  status  of  the  systems  at  the  ‘simple’  level.”  These  sentiments 
come  from  the  implementation  of  the  LISI  construct  for  interoperability,  which  is  based 
upon  detailed  surveys  of  subject  matter  experts  making  it  unwieldy  for  the  average 
program  office  system  engineer  to  implement. 
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As  an  interesting  side  note  to  this  effort,  the  Joint  Staff  has  discontinued  the  required 
use  of  LISI  in  May  2005. 

3.1.6  InspeQtor  Tool 

The  research  on  LISI  brought  forth  the  discovery  of  a  tool  that  MITRE  developed  for 
DISA  which  implements  the  LISI  construct  entitled  InspeQtor.  This  web  based  front-end 
tool  was  designed  to  ease  the  implementation  of  the  LISI  models.  The  heart  of  the 
InspeQtor  tool  is  a  survey  questionnaire  composed  of  hundreds  of  detailed  questions, 
which  are  focused  on  the  infonnation  exchanges  and  technical  infrastructure.  Users  of 
the  tool  must  first  register  on  the  SIPRNET  system  and  then  complete  the  detailed 
questionnaire  by  selecting  the  appropriate  response  on  the  survey  form.  InspeQtor  stores 
and  analyzes  the  answers  and  generates  reports  on  the  systems.  These  reports  are 
available  to  describe  both  the  single  system  and  comparisons  between  multiple  systems. 
Other  users  are  able  to  log  into  the  tool  and  look  at  the  survey  response  for  all  systems  as 
well  as  the  LISI  evaluations.  Users  are  not  allowed  to  modify  existing  survey  responses 
for  systems  they  do  not  own. 

3.2  Enterprise  Architecture  (EA)  Score  Card 

As  an  alternate  means  of  assessment  to  the  Levels  of  Information  System 

Interoperability  (LISI)  Model,  the  Enterprise  Architecture  (EA)  Score  Card  approach  was 
developed  by  Jaap  Schekkerman  of  the  Institute  For  Enterprise  Architecture 
Developments  (IFEAD)  in  the  Netherlands.  This  analysis  tool  is  less  comprehensive 
than  the  LISI  Model  and  does  not  depend  on  interviews  with  subject  matter  experts. 
Surveys  of  systems  can  be  added  to  the  methodology  if  desired.  The  EA  Score  Card  does 
not  exclusively  focus  on  interoperability  between  systems  or  architectures.  However,  this 
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tool  does  provide  a  qualitative  measure  of  EA  quality  and  completeness.  A  primary 
benefit  is  that  it  can  be  used  on  the  Architecture  products  themselves,  before  a  system  has 
been  prototyped  or  constructed.  EA  Score  Card  provides  a  path  for  improvement  in  that 
Systems  or  Chief  Engineers  are  meant  to  further  develop  all  aspect  areas  rated  Partially 
Clear  or  Unclear  until  they  are  rated  Clear  by  later  reviews.  The  results  of  the  EA  Score 
Card  analysis  are  presented  in  a  matrix  called  the  Extended  Enterprise  Architecture 
Framework,  shown  in  Table  5. 


Table  5:  Extended  Enterprise  Architecture  Framework 


Aspect  Area 

Abstraction  Level 

Business 

Information 

Infrastructure 

Contextual  Level 

Business  Goals, 
Drivers  and 
Concepts 

Activities  the 
Business  Performs 

Systems  Goals, 
Drivers  and 

Concepts 

Technology  Goals, 
Drivers  and 

Concepts 

Environmental 

Level 

Extended 

Enterprise  Value 
Net 

Extended 

Enterprise 

Information 

Exchange 

Extended 

Enterprise 

Interoperability 

Extended 

Enterprise 

Interconnection 

Conceptual  Level 

Level  of  Business 
Collaboration 

Level  of 

Information 

Interaction 

Level  of 
Interoperability 

Level  of 
Interconnection 

Logical  Level 

Type  of  Business 
Collaboration 

Type  of 

Information 

Interaction 

Type  of 
Interoperability 

Type  of 
Interconnection 

Physical  Level 

Solutions  of 

Business 

Collaboration 

Solutions  of 
Information 
Interaction 

Solutions  for 
Interoperability 

Solutions  for 
Interconnection 

Transformational 

Level 

Granularity  of 
Change 

Impact  of  Change 

Timeframe  for 
Change 

Timeframe  for 
Change 

3.2.1  EA  Score  Card  Aspect  Areas 

The  EA  Score  Card  examines  four  aspect  areas  of  the  Enterprise  Architecture;  Business, 
Information,  Information  Systems  and  Technology  Infrastructure.  Business  represents 
the  organizational  and  management  processes  in  the  architecture.  Information  measures 
the  information  needs,  flows  and  relationships.  Information  Systems  covers  the 
automated  processing  of  functions.  Technology  Infrastructure  corresponds  to  the 
logistics  and  support  for  the  information  systems  and  their  connections.  The  EA  Score 
Card  then  decomposes  these  four  aspect  areas  into  six  abstract  levels  of  concern; 
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Contextual,  Environmental,  Conceptual,  Logical,  Physical,  and  Transformational,  to 
construct  a  four-by-six  matrix  as  shown  in  Table  5.  The  Contextual  level  describes  the 
scope  and  mission  of  the  organization  and  vision  of  the  architecture.  The  Environmental 
level  focuses  on  the  extended  business  relationships  and  information  flows.  The 
Conceptual  level  explores  the  functional  and  non-functional  requirements,  goals  and 
objectives  of  the  architecture.  The  Logical  level  addresses  solutions  and  sub-functions. 
The  Physical  layer  examines  the  physical  solutions,  concrete  products  and  techniques. 
Finally,  the  Transfonnational  level  measures  the  payoff  in  terms  of  cost,  organizational 
impacts  and  benefits40.  The  EA  Score  Card  tabulates  the  sums  of  each  level  in  an  “All 
Levels”  score  for  each  aspect  area. 

Like  the  Zachman  Framework’s  six  by  five  matrix  shown  in  Figure  7,  the  EA  Score 
Card  assesses  the  Contextual  (equivalent  to  Zachman’s  Planning  Model)  Conceptual, 
Logical,  and  Physical  levels.  However,  the  Zachman  Framework  has  no  cognate  to 
measure  costs,  organizational  benefits  and  security  impacts  as  does  the  EA  Score  Card’s 
Transformational  Level.  The  Zachman  Framework  goes  further  in  exploring  the 
architecture  in  dimensions  of  Time,  People,  and  Motivation  along  with  Data  (Information 
in  the  EA  Score  Card),  Function  (Business)  and  Network  (Information  Systems  and 
Technology  Infrastructure). 
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ZACHMAN  FRAMEWORK 


DATA 

(What) 


TIME 
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PEOPLE 

FUNCTION 

MOTIVATION 
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(How) 

(Why) 

Figure  7:  Zachman  Framework 

The  EA  Score  Card  matrix  identifies  the  undefined  or  nebulous  sectors  of  the 
architectures  as  well  as  thoroughly-documented  sections.  It  can  be  used  to  assess 
whether  the  architecture  fulfills  its  purpose  as  well  as  identify  areas  for  improvement. 
Further,  the  EA  Score  Card  assesses  integration  to  report  the  consistency  of  architecture 
documentation  products.  The  EA  Score  Card  measures  the  architecture  products 
themselves,  so  it  can  assess  how  well  the  knowledge  can  be  transferred  and  maintained. 

3.2.2  EA  Score  Card  Advantages  and  Disadvantages 

The  advantages  of  the  EA  Score  Card  are  its  visual  and  intuitive  layout,  the  fact  it 

measures  each  EA  on  its  own  merit,  and  that  it  can  be  applied  by  Systems  Engineering 
students  without  proprietary  software  or  access  to  classified  systems.  A  sample  EA  Score 
Card  is  shown  in  Figure  8.  The  primary  disadvantage  is  the  tool  does  not  explicitly 


28 


measure  interoperability  between  systems.  A  further  disadvantage  is  non-systems 
engineers  can  misinterpret  the  numerical  results.  For  example,  a  score  of  70%  in  the 
Contextual  Level  does  not  mean  the  architecture  is  “good  enough.”  The  EA  Score  Card 
shows  strengths  and  weakness  in  the  architecture,  not  a  stop  light  bottom-line  assessment. 


Enterprise  Architecture  Score  Card 

Clear  =  Well  defined  and  documented 

Partially  Clear  =  partially  addressed  and  documented 

Unclear  =  NOT  identified  or  addressed,  NOT  defined  or  NOT  documented 

ASC 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Level  of 

Alignment/ 

Integration 

Total  Status 

2 

1 

0 

Questions  to  the  enterprise 
architecture  result 

Business 

Information 

Information 

Technology 

Infrastructure 

Factor  0-2; 

0=lnsufficient 

l=Average 

2=Full 

1 

Are  the  Mission,  Vision,  Goals, 

&  Objectives  of  the  Enterprise 
Architecture? 

2 

Is  the  Scope  of  the  enterprise 
architecture  program? 

3 

Is  the  Form  &  Function  Level 
of  deliverables? 

4 

Is  the  Business  &  IT  Strategy? 

5 

Are  the  Guiding  Principles  & 
Drivers? 

6 

Are  the  Key  Performance 
Indicators? 

7 

Are  the  Critical  Success 

Factors? 

8 

Are  the  Critical  Stakeholders? 

Sub  Score  Contextual  Level 

9 

Are  the  Collaborative  Parties 
Involved? 

10 

Are  the  Contractual 

Agreements? 

11 

Are  the  Interoperability 
standards? 

12 

Are  the  related  Law  & 
Regulations? 

13 

Is  the  Ownership  of 

Information? 

Sub  Score  Environmental  Level 

14 

Are  the  Functional 

Requirements? 

15 

Are  the  Non-Functional 
Requirements? 

16 

Are  the  concepts  in  use? 

17 

Are  the  Security  Requirements? 

18 

Are  the  Governance 
Requirements? 

Sub  Score  Conceptual  Level 

19 

Are  the  deliverables  at  logical 
level? 

20 

Are  the  critical  logical  design 
decisions? 

21 

Are  the  critical  logical  design 
decisions  traceable? 

22 

Are  the  Logical  Description 
Methods  &  Techniques? 

23 

Is  at  logical  level  the  use  of 
Modeling  Tools? 

24 

Are  the  Logical  Standards? 

Sub  Score  Logical  Level 

25 

Are  the  deliverables  at  physical 
level? 

26 

Are  the  critical  physical  design 

29 


decision? 

27 

Are  the  critical  physical  design 
decisions  traceable? 

28 

Are  the  Physical  Description 
Methods  &  Techniques? 

29 

Is  at  the  physical  level  the  use 
of  Modeling  tools 

30 

Are  the  Physical  Standards? 

Sub  Score  Physical  Level 

31 

Critical  Design  Decisions 

32 

Is  the  Organization  Impact? 

33 

Are  the  Costs  Consequences? 

34 

Is  the  Security  Impact? 

35 

Is  the  Governance  Impact? 

Sub  Score  Transformational  Level 

Total  Score  All  Levels 

Figure  8:  The  Enterprise  Architecture  Score  Card  Example 


3.3  Comparison  of  LISI  and  EA  Score  Card 

On  the  surface,  the  LISI  and  EA  Score  Card  models  seem  to  be  very  comparable 

approaches.  However,  this  is  only  true  in  the  separation  of  EA  Score  Card  aspect  areas 
and  LISI  Interoperability  Attributes.  The  EA  Score  Card  analyzes  Business,  Information, 
Information  Systems  and  Technology  Infrastructure  while  the  LISI  analyzes  Procedures, 
Applications,  Infrastructure  and  Data.  The  correlation,  as  depicted  in  Figure  9,  is  not 
always  a  one  for  one  representation.  Areas  such  as  the  LISI  Infrastructure  and  EA  Score 
Card  Technology  Infrastructure  characteristics  are  virtually  identical.  This  is  also  true  for 
the  LISI  Data  and  EA  Score  Card  Information.  On  the  other  hand,  some  of  the  EA  Score 
Card  Aspect  Ares  are  more  of  a  superset  that  includes  the  LISI  Attributes.  The  LISI 
Application  attribute  focuses  predominantly  on  software  while  the  EA  Score  Card 
Information  Systems  includes  both  software  and  hardware.  The  LISI  Procedures  is 
roughly  the  same  as  the  EA  Score  Card  Business  aspect  but  LISI  focuses  more  on  the 
management  policies  and  standards  driving  system  integration  and  requirements 
determination,  where  as  the  EA  Score  Card  Business  aspect  encompasses  on-going 
organizational  functions  beyond  those  necessary  for  acquisition. 
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EA  Score  Card  Aspect  Areas 


LISI  Interoperability 
Attributes 


Figure  9:  Comparison  of  LISI  Interoperability  Attributes  and  EA  Score  Card  Aspect  Areas 

The  scoring  of  the  two  models  is  very  different.  The  EA  Score  Card  reports  on  six 

separate  and  quantized  abstract  levels  of  the  architecture.  It  keeps  each  output  area 
separate  throughout  the  entire  evaluation.  For  example,  a  high  Business  Contextual 
Level  score  is  independent  of  the  Technology  Infrastructure  Physical  Level  results.  On 
the  other  hand,  the  LISI  models  rolls  all  measurements  into  a  level  assessment  for  each 
Interoperability  Attributes.  For  example,  if  a  system  has  a  Domain  Level  3b  score  for 
Procedures  but  only  a  Functional  Level  2c  score  for  Applications.  The  LISI  model 
reports  a  single  final  score  by  reporting  the  lowest  score  of  the  four  Attributes,  a 
Functional  Level  2c. 

The  biggest  difference  is  the  methodology  of  the  two  models.  The  EA  Score  Card  is 
readily  available  from  the  Institute  For  Enterprise  Architecture  Developments  (IFEAD) 
web  site.  A  layman  can  apply  the  Score  Card  directly  to  the  architectural  products  under 
review.  Finally,  the  EA  Score  Card  is  only  35  questions  that  are  answered  with  respect  to 
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four  different  aspect  areas.  By  contrast,  the  LISI  model  and  the  InspeQtor  tool  is  only 
accessible  from  a  Defense  Information  Systems  Agency  (DISA)  SIPRNET  web  site,  with 
a  separate  log-on  to  the  tool.  The  database  questionnaire  is  hundreds  of  questions  in 
more  than  50  categories.  These  questions  were  designed  to  be  answered  by  the  subject 
matter  experts,  requiring  a  level  of  detail  that  is  much  greater  than  from  reviewing  the 
DoDAF  products. 

The  results  of  both  tools  need  to  be  closely  examined  to  get  a  complete  picture  of  the 
assessment.  For  example,  LISI  reports  the  lowest  level  as  the  overall  score,  so  the  other 
areas  the  system  could  score  much  higher.  So  for  both  the  LISI  and  EA  Score  Card  tools, 
the  systems  engineer  must  understand  the  entire  context  of  the  results  in  order  to  make  a 
useful  assessment. 

4  Methodology 

4.1  EA  Score  Card  Methodology 

To  analyze  the  three  target  architectures,  the  approach  was  to  answer  each  of  the  35 
questions  on  the  EA  Score  Card  applied  to  each  of  the  aspect  areas  by  reading  through 
the  views  and  material  until  finding  the  answers  or  determining  the  architects  had  not 
considered  that  aspect  of  the  architecture.  When  an  answer  was  found  in  multiple 
locations  and  view,  the  score  was  increased  for  that  question.  Unfortunately,  the  relative 
subjectivity  of  the  EA  Score  Card  means  each  reviewer  has  to  become  intimately  familiar 
with  the  architecture  during  the  analysis  process.  Additionally,  in  order  to  apply 
consistent  scoring  only  one  reviewer  scored  all  three  architectures.  Multiple  analyses  by 
independent  reviewers  was  considered,  but  due  to  time  constraints  this  is  left  for  future 
research. 
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4.1.1  Assumptions  and  Constraints 

During  the  EA  Score  Card  Analysis,  the  following  assumptions  were  made: 

1 .  The  students  are  not  the  subject  matter  experts  in  any  of  the  three  enterprise 
architectures  or  component  systems  analyzed  (CCIC2S,  C2C  and  GIG). 
Particularly  since  they  did  not  participate  in  the  creation  or  editing  of  any  of 
the  architectures.  This  is  the  fundamental  direction  of  the  effort  and  allows 
for  a  review  by  outsiders. 

2.  Only  the  architectural  material  provided  was  analyzed.  This  included  any 
Object-Oriented  or  Structured  Analysis  created  architectural  views,  diagrams, 
databases  worksheets,  white  papers,  operational  requirements  documents  or 
briefings  included  with  each  architecture.  Additional  material  may  have 
improved  the  scoring,  along  with  updated  versions  of  the  architectures. 

3.  The  rating  was  marked  as  “Clear”  for  any  of  the  35  analysis  questions  if  a 
layman  could  understand  the  architecture  in  each  aspect  area,  “Partially  Clear” 
if  the  question  was  not  fully  answered  or  “Unclear”  if  the  architecture  did  not 
address  the  area  of  concern.  Unclear  was  also  recorded  if  the  architecture  was 
missing  the  necessary  view. 

4.  In  accordance  with  the  EA  Score  Card  methodology,  2  points  were  awarded 
for  each  Clear  rating,  1  for  each  Partially  Clear  and  zero  for  each  Unclear 
rating.41 

5.  EA  Score  Card  questions  asking  “Governance”  concerns  were  treated 
synonymously  as  “management”  concerns. 

6.  The  colors  used  on  the  EA  Score  Card  are  taken  from  the  example  included 
with  Schekkerman’s  original  white  paper. 

4.2  LISI  Methodology  and  Implementation  Issues 

The  original  intent  was  to  apply  the  LISI  model  in  the  same  manner  as  was  applied  to 

the  EA  score  card,  which  looked  at  the  architecture  products  and  answered  the  survey 
questions.  However,  a  number  of  factors  conspired  to  make  this  not  possible.  First, 
access  to  the  InspeQtor  tool  is  controlled  by  the  Defense  Infonnation  Systems  Agency 
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(DISA)  who  recently  moved  the  entire  tool  to  the  SIPRNET  and  increased  the 
classification  of  the  tool,  data  and  products  to  the  secret  level.  This  required  procurement 
of  both  SIPRNET  accounts  and  accounts  on  the  InspeQtor  tool.  The  latter  account  turned 
out  to  be  more  problematic  than  anticipated,  taking  several  weeks  to  gain  access.  The 
second  factor  was  the  inability  to  create  a  separate  workspace  for  inputs  outside  of  the 
formal  tool  database.  In  fact,  the  standard  access  privileges  on  InspeQtor  prevent  non¬ 
system  owners  from  modifying  existing  surveys  on  the  system  and  there  is  not  a  separate 
learning  area  on  the  tool  for  users  to  perform  trial  runs.  A  decision  was  made  to  instead 
look  at  the  existing  data  surveys  on  LISI  for  C2C,  CCIC2S  and  GIG  and  see  how  it 
compares  to  the  EA  Score  Card  results  to  gain  an  insight  on  how  the  two  tools  compare. 
However,  only  CCIC2S  survey  data  was  available  on  the  LISI  System  so  the  system  to 
system  interoperability  examination  intended  with  these  three  architectures  was  not 
possible.  It  was  possible  to  look  at  how  CCIC2S  compares  to  some  other  systems, 
outside  of  the  GIG  and  the  C2C,  and  use  that  as  a  basis  for  understanding  how  LISI 
determines  system  to  system  interoperability.  The  final  consideration  was  to  attempt  to 
complete  the  LISI  survey  based  on  a  review  of  the  architectural  products  for  the  CCIC2S. 
This  proved  unfeasible  due  to  the  detailed  nature  of  the  LISI  surveys,  which  are  very 
technical  standards  and  operating  system  specific.  Further,  these  surveys  are  not 
compatible  with  the  level  of  detail  in  the  architectures.  For  example,  the  LISI  tool  has  63 
separate  questions  concerning  the  applicable  security  standards  for  the  system.  In  other 
words,  the  enterprise  architectures  were  at  a  higher  level  (enterprise)  while  the  intent  of 
LISI  is  to  examine  systems  at  the  detailed  system  interface  level.  After  discussing  the 
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prospects  of  completing  a  survey,  it  was  detennined  to  exceed  the  time  and  scope  of  this 
project. 

5  Results  of  Analysis 

5.1  EA  Score  Card  Analysis 

The  results  of  the  analysis  of  the  three  architectures  using  the  EA  Score  Cards  are 
presented.  This  analysis  was  completed  in  April  -  May  2005.  The  EA  Score  Card  model 
simply  presents  the  raw  numbers  as  a  final  product.  The  raw  data  sheets  from  the 
analysis  can  be  found  in  Appendix  A.  In  order  to  succinctly  present  the  results,  tabulated 
the  totals  for  each  aspect  area  and  level  are  shown  along  with  graphical  bar  charts  to 
compare  levels  and  spot  trends.  Graphic  representation  of  results  is  provided  using  the 
same  scale  height,  with  the  highest  percentage  being  the  baseline,  to  minimize 
misinterpretation. 

To  ensure  comparison  of  only  like  characteristics,  each  architecture  was  measured  for 
a  consistency  across  the  six  abstract  levels  plus  the  all  levels  score.  To  develop  this 
consistency,  a  standard  deviation  of  the  level  scores  was  used  to  show  the  degree  of 
completeness  for  each  architecture.  Consistency  is  defined  as  a  less  than  10%  range 
among  aspect  area  scores  for  the  same  level  of  concern.  The  limitation  of  this  yardstick 
is  each  architecture  must  be  compared  to  the  stated  purpose  for  creating  it  in  the  first 
place. 


35 


5.1.1  C2C  EA  Score  Card  Analysis 


Table  6:  C2C  Results 


Business 

Technology 

Information  Infrastructure 

Contextual  Level 

44% 

44% 

50% 

50% 

Environmental  Level 

70% 

80% 

70% 

70% 

Conceptual  Level 

70% 

60% 

60% 

60% 

Logical  Level 

50% 

42% 

42% 

42% 

Physical  Level 

8% 

17% 

17% 

17% 

Transformational  Level 

50% 

10% 

10% 

20% 

All  Levels 

47% 

42% 

42% 

43% 

Standard  Deviation 

.74 

.90 

.87 

.82 

Table  7:  C2C  Predominate  Views  Used  in  Score  Card  Analysis 


Contextual  Level 

AV-1  Overview  and  Summary  Information 
OV-1  High  Level  Operational  Concept  Graphic 
OV-3  Operational  Information  Exchange  Matrix 
OV-6a  Operational  Rules  Model 

Environmental  Level 

AV-2  Integrated  Dictionary 
OV-3  Operational  Information  Exchange  Matrix 
OV-5  Operational  Activity  Model 
OV-6a  Operational  Rules  Model 
SV-4  Systems  Functionality  Description 

Conceptual  Level 

OV-3  Operational  Information  Exchange  Matrix 
OV-5  Operational  Activity  Model 
OV-6a  Operational  Rules  Model 
SV-4  Systems  Functionality  Description 

Logical  Level 

AV-1  Overview  and  Summary  Information 
OV-5  Operational  Activity  Model 
SV-5  Operational  Activity  to  Systems  Function  Matrix 
TV-2  Technical  Standards  Forecast 

Physical  Level 

SV-4  Systems  Functionality  Description 
TV-2  Technical  Standards  Forecast 

Transformational  Level 

AV-1  Overview  and  Summary  Information 
SV-9  Systems  Technology  Forecast 
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C2C  Results 
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Figure  10:  C2C  Score  Card  Results 


The  C2C  analysis  required  the  most  time  and  energy  due  to  a  lack  of  a  webpage 
browser  or  central  directory  to  guide  us  through  the  architecture.  Further,  the  C2C 
Overview  briefing  from  December  2003  raised  the  question  of  whether  an  architecture 
completed  at  the  Unclassified  level  contained  sufficient  detail  or  thoroughness  to  warrant 
the  effort  to  complete  it.  A  cohesive  assessment  of  the  C2C  Architecture  did  not  come 
into  focus  until  the  data,  in  Table  6,  was  graphically  displayed  as  shown  in  Figure  10. 

The  C2C  Architecture  rated  the  highest  scores,  meaning  it  was  most  fully 
documented,  at  the  Environmental  and  Conceptual  levels,  due  to  its  2005  “As-Is”  and 
2012  “To-Be”  views.  A  complete  listing  of  the  architectural  products  used  in  the  analysis 
is  in  Table  7.  Specifically,  the  architects  created  detailed  OV-5  Operational  Activity 
Models  and  OV-6a  Operational  Rules  Models  for  both  2005  and  2012  timeframes  as  well 
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as  stand-alone,  focused  models  to  support  the  Time  Critical  Targeting  concept  and  its 
evolutionary  successor  Time  Sensitive  Effective  Operations. 

All  four  aspect  areas  were  consistent  for  each  level  of  concern,  save  one.  Using  the 
definition  of  consistency  as  less  than  a  10%  range  among  aspect  area  scores  for  the  same 
level  of  concern,  there  was  a  great  disparity  between  the  Business  Aspect  Area  (50%)  and 
the  other  three  areas  (10%  -  20%)  in  the  C2C  Transformational  Level.  This  is  due  to 
relatively  clear  depictions  of  decision  processes,  milestones,  organizational  impacts  and 
costs  for  the  C2C  Business  processes  in  the  AV-1  Overview  and  Summary,  C2C 
Overview  Briefing  and  Program  Plan  Documents.  The  Information,  Infonnation  Systems 
(IS)  and  Technology  Infrastructure  (TI)  impacts  were  merely  mentioned  in  one  slide  of 
the  C2C  Overview  Briefing  or  not  at  all. 

Typical  to  all  three  architectures,  the  Physical  Level  scored  the  lowest,  showing  the 
lack  of  views  prepared  for  physical  systems.  This  may  be  a  necessary  by-product  of 
investigating  big-picture  enterprise  architectures  which  become  too  ponderous  to  keep 
current  if  they  explicitly  define  the  physical  levels  of  the  systems  which  reside  within 
their  macro-level  system. 

Based  on  the  EA  Score  Card  analysis,  the  C2C  Architecture  partially  met  its  many 
purposes.  The  Architecture  does  a  commendable  job  of  supporting  C2C  definition, 
conceptual  design,  requirements  development,  early  milestone  decision  reviews  and 
concept  of  operation  maturation  as  shown  by  strong  Environmental  and  Conceptual  Level 
scores.  Its  Contextual  and  Logical  levels  scores  seem  adequate  to  facilitate  Global 
Information  Grid  architecture  compliance.  The  C2C  architecture  infers  Integration  Points 
with  the  GIG  Architecture  by  mentioning  GIG  compliance  as  an  objective  in  one  of  the 


38 


many  briefings  included  with  the  architecture,  however  it  does  not  graphically  illustrate 
any  Integration  Points  anywhere  in  the  architecture. 

However,  for  an  architecture  created  to  prototype  a  Joint  Mission  Thread  and  Air 
Force  Mission  Area,  one  might  expect  higher  Transformational  Level  and  All  Level 
Scores.  Also,  one  would  hope  for  stronger  architectural  views  that  rely  less  on 
supporting  briefing  material  to  avoid  the  perception  of  “architecting  by  Power  Point.” 
This  weakness  can  be  explained  by  the  fact  the  C2C  version  2.0  is  a  draft  architecture42 
and  the  next  revision,  expected  late  2005,  should  be  more  robust. 


5.1.2  CCIC2S  EA  Score  Card  Analysis 


Table  8:  CCIC2S  Results 


Business 

Technology 

Information  Infrastructure 

Contextual  Level 

50% 

89% 

89% 

67% 

Environmental  Level 

40% 

60% 

50% 

70% 

Conceptual  Level 

50% 

70% 

80% 

60% 

Logical  Level 

25% 

58% 

33% 

33% 

Physical  Level 

0% 

8% 

8% 

0% 

Transformational  Level 

20% 

10% 

10% 

10% 

All  Levels 

32% 

53% 

49% 

42% 

Standard  Deviation 

.75 

.94 

.93 

.87 

Table  9:  CCIC2S  Predominate  Views  Used  in  Score  Card  Analysis 


Contextual  Level 

OV-4  Operational  Command  Relationships 
OV-5  Operational  Use  Cases 

Environmental  Level 

Operational  Requirements  Document 

Conceptual  Level 

Operational  Requirements  Document 

Logical  Level 

OV-5  Activity  Diagrams 

Physical  Level 

Operational  Requirements  Document 

Transformational  Level 

White  Paper 
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CCIC2S  Results 


Figure  11:  CCIC2S  Score  Card  Results 


Unlike  the  other  two  enterprise  architectures  examined,  the  AFSPC  systems 
engineers,  architects,  and  requirements  analysts  used  an  object-oriented  approach, 
commonly  used  by  software  engineers,  to  rapidly  update  the  evolving  missions  of  the 
CCIC2S  architecture.  Initially  a  Structured  Analysis  approach  was  tried,  but  it  was 
abandoned  as  it  could  not  keep  pace  with  the  mission  evolution.43  They  created  an 
operational  architecture  heavily  focused  on  the  Information  and  Information  Systems  (IS) 
Aspect  Areas. 

The  raw  data  is  presented  in  Table  8,  with  the  graphical  representation  in  Figure  11. 
Consistency  varied  the  most  among  the  CCIC2S  aspect  areas.  By  using  the  Consistency 
definition  of  a  10%  or  less  variance  in  the  range  of  scores,  only  the  Physical  and 
Transformational  Levels  are  consistent  for  the  CCIC2S  Architecture.  They  are 
consistently  low,  however,  averaging  roughly  10%.  They  highlight  a  general  lack  of 
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deliverables,  modeling  tools,  impacts  and  cost  development  for  the  architecture.  Physical 
concerns  are  not  addressed  at  all  for  the  Business  and  Technology  Infrastructure  aspect 
areas.  This  is  reflected  in  the  All  Levels  scores.  The  lower  scores  in  the  Logical  and 
Physical  Levels  is  also  attributable  to  the  fact  the  CCIC2S  operational  architecture 
contains  no  Systems  Views.  This  is  concerning  since  CCIC2S  was  characterized  as  more 
‘system-like’  than  ‘enterprise.’  However,  it  is  necessary  to  recall  the  purpose  for  which 
the  architecture  is  built.  CCIC2S  is  an  operational  architecture  put  together  by  an 
operational  command  for  the  purpose  of  capturing  user  requirements  leading  to  a  detailed 
design,  so  one  would  not  expect  to  find  many  Systems  Views.  A  complete  list  of  the 
products  used  in  this  assessment  is  found  in  Table  9. 

Information  and  IS  scores  are  higher  than  Business  and  Technology  Infrastructure 
(TI)  across  most  levels  of  concern.  This  is  due  to  the  deeply-comprehensive  OV-5  Use 
Cases  which  present  a  vast  range  of  operational  functions,  and  then  cross-references  them 
to  the  resulting  Operational  Requirements  Document  (ORD).  The  Contextual 
Information  and  IS  scores  are  the  highest  in  the  entire  analysis  at  89%  each,  signifying 
the  architecture  is  most  fully  developed  for  the  enterprise  mission,  drivers,  vision  and 
scope.  This  also  is  reflected  in  the  All  Levels  Scores  as  well. 

The  CCIC2S  Architecture  met  its  purpose,  because  it’s  primary  function  was  to 
capture  user  and  operator  requirements  for  detailed  design  and  test  of  the  CCIC2S 
system.  However,  additional  systems  functions  need  to  be  identified  before  testing  can 
begin.  The  operational  architecture  needed  to  integrate  common  functions  across  many 
different  legacy  systems  developed  in  stovepipe  manner.  The  resulting  architecture 
contains  the  deepest  levels  of  operational  functional  details  for  the  ORD  of  all  three 
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architectures  analyzed.  However,  the  object-oriented  approach  resulted  in  complex  use 
cases  that  relied  upon  the  ORD  for  navigation  and  indexing.  Only  systems  engineers 
familiar  with  both  Structured  Analysis  and  object-oriented  approaches  will  feel 
comfortable  reading  such  difficult  diagrams.  Systems  views  should  be  developed  for  the 
architecture  to  capture  a  complete  view  of  the  proposed  capability.  Integration  Points  are 
not  identified  to  show  commonality  with  other  architectures,  although  GIG  interfaces  are 
mentioned  briefly  at  the  beginning  of  the  Operational  Requirements  Documents  (ORD). 


5.1.3  GIG  EA  Score  Card  Analysis 


Table  10  GIG  Results 


Business 

Technology 

Information  11  Systems  Infrastructure 

Contextual  Level 

72% 

72% 

67% 

56% 

Environmental  Level 

50% 

50% 

60% 

40% 

Conceptual  Level 

70% 

80% 

80% 

60% 

Logical  Level 

75% 

67% 

75% 

75% 

Physical  Level 

25% 

42% 

50% 

50% 

Transformational  Level 

60% 

60% 

60% 

60% 

All  Levels 

60% 

63% 

65% 

57% 

Standard  Deviation 

.68 

.70 

.71 

.70 
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Table  11  GIG  Predominate  Views  Used  in  Score  Card  Analysis 


Contextual  Level 

OV-1  High  Level  Operational  Concept  Graphic 
OV-2  Operational  Node  Connectivity 
OV-4  Command  Relationships 
SV-1  Systems  Interface 
SV-2  Systems  Communications 

Environmental  Level 

Program  Management  Plan 

Conceptual  Level 

AV-1  Overview  and  Summary  Information 

Logical  Level 

OV-3  Operational  Information  Exchange  Requirements 

OV-5  Activity  Model 
SV-1  Systems  Interface 
SV-2  Systems  Communications 
SV-3  System  Matrix 
SV-4  System  to  System  Functions 

Physical  Level 

TV-2  Technical  Standards  Forecast 

Transformational  Level 

Program  Management  Plan 
Capstone  Requirements  Document 

GIG  Results 


Figure  12:  GIG  Score  Card  Results 


The  HTML  browser  developed  for  the  GIG  v2  Architecture  was  both  the  easiest  to 
use  layout  and  contained  the  most  clearly  labeled  architectural  products.  The  GIG  scores, 
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shown  in  Table  10  and  graphically  in  Figure  12,  are  both  the  highest  and  most  consistent 
both  within  aspect  areas  and  across  levels  of  concern.  This  is  easily  explained  by  the 
nearly  complete  sets  of  architectural  views,  listed  in  Table  11,  developed  for  the  three 
Strategic,  Operational  and  Tactical  Use  Case  scenarios.  As  with  the  other  two 
architectures,  the  Physical  Level  has  the  lowest  scores,  representing  the  least  amount  of 
documentation.  However,  the  GIGs  Physical  Level  scores  were  comparable  to  the  more 
developed  levels  of  the  other  two  architectures.  The  GIG  v2  Architecture  does  not  need 
well  developed  Physical  Level  models  to  facilitate  Chief  of  Information  Operations 
(CIO)  planning  and  acquisition  decisions. 

The  Conceptual,  Logical  and  Contextual  Levels  scored  the  highest  percentages; 
showing  well-developed  missions,  vision,  requirements,  logical  solutions  and  scope  of 
the  EA.  The  Transformational  and  Environmental  Levels  were  slightly  lower  but  still 
above  the  comparable  levels  of  the  other  two  architecture.  The  Transformational  Levels 
was  completely  consistent  across  aspect  areas  (all  four  scored  60%).  This  shows  the 
developers  efforts  to  make  the  operational,  systems  and  technical  views  equally  strong.  It 
also  ties  back  to  the  architecture’s  purpose  to  serve  as  a  NCW  model.  Additional  views 
to  support  the  Transfonnational  Level  will  only  improve  the  GIG  architecture. 

The  GIG  v2  architecture  meets  it  purpose,  providing  the  initial  architectural 
description  of  NCOW  concepts,  tenninology  and  modeling.  The  vast  scope  of  the  GIG 
makes  defining  the  Environmental  and  Physical  Levels  nearly  impossible.  The  use  of  the 
Strategic,  Operational  and  Tactical  Use  Cases  makes  the  crisp  architectural  views  able  to 
present  the  knowledge  without  having  to  rely  on  extensive  briefing  material,  operational 
requirements  documents  or  white  papers.  Integration  Points  for  the  GIG  v2  Architecture 
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are  defined  with  national-level  strategic  decision  makers  as  nodes  on  the  OV-1  High 
Level  Operational  Graphic.  Integration  Points  with  other  specific  Enterprise  Architecture 
are  not  explicitly  defined.  Additional  architectural  views  would  definitely  improve  the 
scores  of  the  GIG  architecture.  However,  one  can  imagine  the  expensive  cost  of 
maintaining  this  easy  to  use,  standardized  document.  The  current  number  of  views  may 
be  the  optimal  for  effective  updates  to  the  GIG  architecture. 

5.2  LISI  Analysis 

The  results  of  the  analysis  of  the  CCIC2S  Architecture  using  the  LISI  model  and 
InspeQtor  front  end  tool  was  conducted  in  May  of  2005.  Much  of  the  data  and 
comparisons  remain  classified,  so  discussion  of  results  in  this  forum  is  limited.  The 
surveys  require  inputs  from  those  knowledgeable  in  the  system  and  not  designed  for  the 
intended  aim  of  conducting  an  assessment  from  only  the  architecture  product.  For 
example,  one  series  of  questions  in  the  survey  asks  about  the  operating  systems  being 
used,  such  as  Linux,  Sun  Solaris  or  Microsoft  Windows  and  the  associated  patches  in 
place. 

However,  the  actual  charts  and  comparison  features  of  the  tool  were  intuitive  to  use, 
quickly  learned  and  easy  to  navigate  using  only  a  standard  internet  browser.  In  looking  at 
one  product  for  examining  interoperability  with  other  systems,  the  tool  displayed  an 
Interconnectivity  Report  in  matrix  form.  While  this  seems  logical,  further  research 
revealed  there  was  no  scientific  rationale  or  logical  connections,  but  rather  the  Subject 
Matter  Experts  claiming  connections  to  other  systems.  It  did  provide  the  ability  to  see  if 
the  systems  experts  claimed  interconnectivity  and,  more  importantly,  if  the  systems 
experts  from  both  systems  agreed  on  the  interconnectivity. 


45 


6  Results  of  System  to  System  Analysis 

6.1  EA  Score  Card  Comparisons 

While  the  EA  Score  Card  model  is  successful  at  identifying  the  holes,  areas  of 

concern  and  strong  points  of  individual  enterprise  architecture,  it  does  not  suggest  ways 
to  compare  architectures  against  each  other.  Neither  does  it  attempt  to  develop  criteria 
for  defining  when  architectures  are  comparable  to  each  other.  Investigation  of  the  three 
architectures  revealed  that  while  the  CCIC2S  and  C2C  Architectures  both  refer  to  the 
GIG  Architecture  in  general  terms,  the  GIG  v2  naturally  does  not  mention  either  of  the 
others.  The  GIG  v2  scenarios  do  not  include  application  of  the  other  architectures  and  it 
was  not  possible  to  develop  a  common  scenario  due  to  a  lack  of  mission  commonality 
between  the  CCIC2S  and  C2C.  The  C2C  is  designed  for  major  theater  war  operations 
while  the  CCIC2S  focuses  specifically  on  AFSPC,  STRATCOM  and  NORAD  space 
missions.  However,  it  is  possible  to  do  a  comparison  at  the  individual  levels  of  concern: 
contextual,  environmental,  conceptual,  logical,  physical,  transformational  and  all  levels; 
as  a  measure  of  interoperability. 
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Comparison  of  Contextual  Level 


□  Business 
B  Information 
B  Info  Systems 

□  Tech-Infrastructure 


Figure  13:  Comparison  of  Contextual  Levels 

As  previously  stated  during  the  CCIC2S  results,  it  showed  the  highest  scores  for  the 
Contextual  Level,  especially  for  the  Information  and  Information  Systems  aspect  areas. 
This  spike  is  not  reflected  in  the  CCIC2S  All  Levels  results.  This  demonstrates  the  pitfall 
of  attempting  to  reduce  the  EA  Score  Card  to  a  simple  average  of  all  levels  and  aspect 
areas.  The  EA  Score  Card  model  is  too  complex  to  reduce  to  a  simple  average  score. 

The  CCIC2S  operational  architecture  was  created  to  support  an  Operational 
Requirements  Document  and  its  low  scores  in  other  levels  does  not  prevent  it  from 
accomplishing  its  fundamental  mission.  The  percentages  for  the  three  architectures  at  the 
Contextual  Level,  as  shown  in  Figure  13,  reflect  the  separate  uses  each  was  created  to 
serve. 
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Comparison  of  Environmental  Levels 


□  Business 

■  Information 

■  Info  Systems 

□  Tech-Infrastructure 


Figure  14:  Comparison  of  Environmental  Levels 

The  data,  in  Figure  14,  shows  the  C2C  has  higher  scores  on  the  Environmental  Level 
due  to  stronger  documentation  of  interoperability  standards,  laws  and  regulations  and 
information  ownership  definitions.  The  C2C  Architecture  is  not  only  stronger  at  the 
Environmental  Level  but  more  consistent  as  well. 

While  the  Environmental  Level  contains  the  bulk  of  interoperability  questions,  it 
cannot  alone  answer  the  question  of  how  interoperable  the  architectures  are.  The  nature 
of  the  architectures  mission  must  be  considered  in  order  to  answer  how  interoperable  the 
architecture  is  as  a  result.  While  both  the  CCIC2S  and  C2C  architectures  define  their 
interfaces  with  the  GIG,  the  reverse  is  not  true. 


Further,  the  CCIC2S  and  C2C  architectures  do  not  directly  interface  due  to  a  lack  of 
common  mission  areas.  This  makes  direct  comparison  significantly  more  difficult  since 
it  is  not  practical  to  develop  a  common  scenario  to  demonstrate  the  interoperability  of 
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integration  points  across  the  enterprise  architectures.  Careful  selection  of  architectures 
and  their  external  system  boundaries  must  be  undertaken  in  order  to  use  scenario 
development  to  illustrate  interoperability.  This  effort  should  be  the  basis  for  another 
group  research  project. 


Comparison  of  Conceptual  Levels 


Figure  15:  Comparison  of  Conceptual  Levels 


Figure  15  displays  the  Conceptual  Level  comparison  where  the  three  architectures 
show  approximately  equivalent  scores  due  to  a  roughly  equal  focus  on  Security, 
Governance,  Functional  and  Non-Functional  Requirements.  The  definition  of 
requirements  is  a  common  objective  to  all  three  architectures  and  is  very  well- 
documented  in  both  the  CCIC2S  and  C2C  architectures.  The  structured  analysis 
approach  used  in  the  GIG  and  C2C  architectures  led  to  OV-5  Activity  Model  views  that 
more  clearly  showed  the  functional  requirements  when  compared  to  the  CCIC2S  object- 
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oriented  UML  produced  OV-5  Activity  Diagram  and  Use  Cases.  The  CCIC2S  architects 
balanced  this  with  a  clearly  defined  requirements  matrix  in  the  ORD. 


Comparison  of  Logical  Levels 


□  Business 

■  Information 

■  Info  Systems 

□  Tech-Infrastructure 


Figure  16:  Comparison  of  Logical  Levels 

At  the  Logical  Level,  depicted  in  Figure  16,  the  GIG  architecture  is  clearly  more 
well-developed  and  consistent.  This  is  due  to  the  clear  and  extensive  Systems  View 
Diagrams  that  expand  on  the  operational  views  functions  and  nodes  and  define  the 
system  entities  that  make  up  a  real  system  of  system.  It  is  also  the  first  level  that 
demonstrates  the  all-around  strength  of  the  GIG  v2  Architecture.  By  comparison,  the 
Object  Orientated  Approach  based  CCIC2S  architecture  contained  only  one  systems 
view.  The  C2C  system  views  only  supported  their  operational  views  and  did  not  include 
an  SV-1  Systems  Interface  or  SV-2  Systems  Communication  view. 
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90% 


Comparison  of  Physical  Levels 


Figure  17:  Comparison  of  Physical  Levels 


While  the  Physical  Levels,  graphed  in  Figure  17,  do  represent  the  least  developed 
level  of  concern  amongst  the  three  subject  enterprise  architectures,  the  GIG  v2  shows  an 
order  of  magnitude  difference  relative  to  C2C  and  CCIC2S.  While  the  GIG  Architecture 
is  weakest  at  the  Physical  Level,  it  is  at  least  Partially  Clear  in  many  areas  the  other  two 
architectures  do  not  even  mention.  This  is  the  second  level  of  concern  that  shows  the  all- 
around  strength  of  the  GIG  Architecture.  Ultimately,  weak  physical  levels  are  expected 
when  reviewing  Enterprise  Architectures  due  to  their  concern  with  the  broad  scope  and 
vision  of  the  subject  organization.  Despite  the  expected  low  scores,  it  is  important  to 
assess  the  physical  aspects  of  the  Enterprise  Architecture  to  provide  systems  engineers 
with  a  complete  understanding  of  their  architecture. 
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Comparison  of  Transformational  Levels 


Figure  18:  Comparison  of  Transformational  Levels 

The  Transformational  Levels  of  Figure  18,  like  the  Physical  Levels  of  Figure  17, 
shows  an  order  of  magnitude  difference  between  the  GIG  Architecture  and  the  CCIC2S 
and  C2C.  Only  the  Business  Aspect  Area  of  the  C2C  is  comparable,  due  mainly  to  a 
definition  of  cost  and  organizational  impacts  in  the  C2C  Overview  briefing.  Risk 
analysis,  cost,  organizational,  management  and  security  impacts  are  the  prime 
determiners  of  Transformational  Level  scores.  In  these  three  cases,  Structured  Analysis 
and  Object  Orientated  approaches  do  not  address  the  transformational  impacts  in 
architectural  views.  This  is  not  a  fault  of  either  approach,  but  rather  of  the  detail  and 
purpose  of  the  individual  architectures.  Transformational  Level  information  was  detailed 
in  supplemental  text  documents  such  as  white  papers,  management  plans  or  the  CCIC2S 
ORD. 
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Comparison  of  All  Levels 


□  Business 

■  Information 

■  Info  Systems 

□  Tech-Infrastructure 


Figure  19:  Comparison  of  All  Levels 

Unfortunately  direct  comparison  of  the  All  Levels  scores  in  Figure  19  only  suggests 
the  all-around  strength  of  the  GIG  v2  Architecture  and  masks  the  successful  portions  of 
all  three  architectures.  This  argues  strongly  against  using  the  All  Scores  percentages  as  a 
“Traffic  Light”  chart  to  show  architecture  merit  in  overly  quick  terms  of 
Green/Yellow/Red.  In  fact,  Systems  Engineers  would  profit  from  NOT  publishing  the 
Score  Cards  themselves  and  presenting  the  strong,  weak  and  missing  parts  of  their 
architectures  in  text  form  to  policy  and  decision  makers.  The  Score  Card  model  requires 
an  in  depth  understanding  of  both  the  architecture  and  the  model  methodology  to 
appreciate  the  raw  data.  There  is  a  risk  non-Systems  Engineers  could  misinterpret  the 
All-Levels  scores  as  being  “good  enough  for  government  work”  without  realizing  their 
architecture  fails  to  serve  one  of  its  fundamental  purposes  because  the  numerical  score 
masks  the  omission.  However,  the  Score  Cards  do  serve  a  valuable  purpose  to  Systems 
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Engineers  in  helping  complete  the  architecture  and  this  merit  makes  it  worthwhile  to 
complete  the  analysis. 


6.1.1  Conclusions 

From  the  percentages  charted  above,  one  can  see  the  relative  comparisons.  The  GIG 
Architecture  scored  highest  across  all  four  aspect  areas  and  the  six  levels  of  concern. 
From  this  analysis,  one  infers  the  GIG  architecture  is  the  most  complete  and  contains  the 
deepest  levels  of  detail.  However,  as  discussed  in  the  individual  results  sections,  all  three 
architectures  have  strengths  and  weaknesses  that  affect  performance  at  their  intended 
uses. 

The  purpose  and  intended  use  of  the  enterprise  architecture  is  still  paramount  in 
measuring  its  effectiveness.  The  All  Levels  results  suggest  the  C2C  and  CCIC2S  are 
roughly  comparable  to  each  other  in  effectiveness  and  completeness,  however,  the 
CCIC2S  architecture  was  only  intended  to  support  an  Operational  Requirements 
Document,  a  goal  it  clearly  achieved  by  careful  examination  of  the  Score  Card  numbers 
and  the  Contextual  and  Conceptual  Level  results.  The  C2C  Architecture  is  described  by 
its  AV-1  Overview  as  a  “Sub-Enterprise  Level”  architecture  to  define  the  highest-level 
aspects  of  the  C2  Constellation  but  does  not  include  all  of  the  functionalities  and 
associated  systems.  Future  phases  of  the  architecture  will  provide  further  depth  and 
extend  the  scope. 

Finally,  the  EA  Score  Card  model,  while  useful  for  assessing  the  completion  and 
maintainability  of  an  enterprise  architecture,  does  not  sufficiently  answer  the  goal  of 
defining  interoperability  among  architectures.  More  detailed  analysis  and  modeling  is 
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required  to  answer  this  question.  In  fact,  the  lack  of  a  readily  useful  tool  to  measure 
interoperability  suggests  applied  research  should  be  undertaken  to  develop  one. 

6.2  LISI  Results  and  Conclusions 

Looking  at  the  specific  data  available  for  CCIC2S,  brings  forth  similar  results  to  the 
analysis  from  the  EA  Score  Card  when  looking  at  only  the  single  architecture.  The 
attributes  of  procedures  and  data  rated  high  while  the  application  and  infrastructure  was 
lower.  This  is  to  be  expected,  and  verifies  the  EA  Score  Card  results,  as  the  CCIC2S 
architecture  was  designed  to  produce  an  Operational  Requirements  document  and  is 
lacking  many  of  the  systems  views  associated  with  a  physical  architecture 

6.3  Findings 

After  having  looked  at  the  Enterprise  Architectures  this  past  year  and  specifically  as 
part  of  this  capstone  project  looking  at  interoperability,  lead  to  arrival  upon  four  key 
findings.  First  and  foremost,  when  evaluating  an  architecture,  the  decision  maker  needs 
to  be  aware  of  the  intended  purpose  for  which  the  architecture  was  built.  This  purpose 
will  heavily  influence  what  architectural  views  are  developed  and  the  depth  of  detail  in 
those  views. 

Second,  not  all  architectures  are  the  same.  For  example,  two  of  the  architectures 
examined  were  intended  for  more  of  an  enterprise/concept  definition  use  (C2  and  GIG), 
thus  did  not  contain  as  much  system  specific  infonnation.  CCIC2S,  on  the  other  hand,  is 
a  little  lower  level  and  has  much  more  of  a  system  feel.  In  addition,  because  of  the 
interactive  nature  of  architectures  the  maturity  of  the  architecture  needs  to  be  kept  in 
mind  when  examining  for  details. 
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The  third  finding  is  the  lack  of  a  single  tool  to  measure  interoperability  based  solely 
on  architectures  documents.  The  Enterprise  Architecture  Score  Card  is  a  good  common 
reference  point  for  systems  engineers  to  start  examining  single  architectures.  During  the 
capstone  project  of  examining  interoperability,  it  was  found  that  the  EA  Score  Card  is 
better  suited  for  evaluating  architectures  in  relation  to  intended  purpose  and  helping  to 
identify  possible  gaps  in  the  architecture.  LISI,  on  the  other  hand,  is  better  suited  for 
identifying  interoperability  issues,  but  is  extremely  difficult  to  use.  To  get  the  complete 
picture,  both  tools  should  be  utilized  as  part  of  a  GAP  analysis  in  the  JCIDS  process,  or 
another  tool  should  be  developed. 

Lastly,  correctly  created  architectures  can  help  the  JCIDS  and  PPBE  processes  for 
making  investment  decisions,  but  because  so  many  architectures  are  incomplete  and 
created  with  different  purposes  it  is  possible  to  understand  why  decision  makers  may  not 
be  realizing  the  perceived  benefits.  The  DoD  did  attempt  to  include  this  information  as  a 
part  of  its  Net-Centric  Program  Assessment.  However,  the  series  of  68  questions  that 
were  a  part  of  the  survey  were  extremely  broad  in  nature.  For  example,  the  survey  asked 
about  the  use  of  architectures  via  conformance  to  the  DoDAF  and  if  the  various  views 
were  updated.  However,  it  did  not  go  into  any  analysis  of  the  architectures  or  integration 
between  systems.  While  this  assessment  doesn’t  measure  interoperability  like  LISI  or  the 
EA  Score  Card,  it  does  attempt  to  re-enforce  conformance  to  net-centric  services  and 
provides  a  use  of  architectures  in  the  budgeting  process. 

The  bottom  line  is  current  enterprise  architectures  assessment  tools  have  not  kept 
pace  with  the  relevant  key  performance  parameters  (KPP).  Over  the  past,  the  LISI  Model 
was  designed  to  perform  the  Interoperability  assessment,  as  shown  in  Figure  20. 
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1998  -  2003: 


Interoperability 

KPP 


Levels  of  Information 
System  Interoperability  (LISI) 


System-to-System  Focus 
Built  for  C4ISR  Framework 
Based  on  SME  Inputs 


Assessment 
■  Finite  Score 


Figure  20:  Past  Interoperability  Assessment  Methodology 


However,  Network  Readiness  is  now  the  dominant  KPP  with  interoperability  just  a 
factor  in  the  NR-KPP  equation.  Likewise  the  assessment  tools  and  methodology  for 
enterprise  architecture  assessment  models  need  to  adapt.  Currently,  there  is  not  a  single 
tool  to  assess  the  quality  of  net-centricity  in  a  system,  as  shown  in  Figure  21. 


2005: 


Network-Centric 

KPP 


Adapting  to  DoDAF 
How  do  we  measure? 


Figure  21:  Today's  Net-Centric  Assessment  Methodology 


The  focus  of  measuring  interoperability  may  no  longer  be  the  question,  but  rather 
how  a  measurement  of  network  centricity  or  network  readiness  compared  to  the  GIG 
standard  as  depicted  in  Figure  22. 
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2007 


Network-Centric 


KPP 


Levels  of  Interoperability  and 
Network  Centricity  (LINC) 


System  to  GIG  Focus 


Evolutionary  DoDAF  structure 


Assessment 

J  >  Finite  Score 
Areas  for 
Improvement 


Based  on  Architecture  Products 


Figure  22:  Proposed  Net-Centric  Interoperability  Methodology 

Completing  the  EA  Score  Card  for  a  few  architectures  provides  the  ability  to  start 
predicting  EA  Score  Card  scores  based  on  the  stated  purpose  and  available  architectural 
views.  This  still  does  not  accomplish  the  goal  of  measuring  system  to  system 
interoperability,  so  additional  research  is  necessary  to  develop  an  assessment  tool  that  can 
measure  not  only  interoperability,  but  more  specifically  the  Network  Centricity  of 
systems  connecting  to  the  GIG. 

6.4  Recommendations  for  Further  Study 

During  the  course  of  this  project,  several  additional  areas  of  interest  were  identified, 

where  further  study  could  be  of  value  to  decision-makers,  system  engineers,  and 
architecture  communities. 

6.4.1  New  Tool  for  Measuring  Interoperability 

The  lack  of  a  readily  applicable  tool  for  measuring  interoperability  between 

architectures,  as  well  as  a  stand-alone  assessment,  suggests  additional  research  and  the 
creation  of  new  tools.  Two  candidate  system  architectures  that  could  be  used  as  control 
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cases  during  tool  development  are  the  CCIC2S  and  the  Rapid  Attack  Identification 
Detection  Reporting  System  (RAIDRS).  Both  of  these  systems  have  completed  LISI 
evaluations  and  the  data  is  readily  available  on  InspeQtor  which  can  be  used  to  validate 
development.  The  task  is  to  develop  a  tool  that  measures  the  net-centricity  of  systems  in 
relation  to  the  GIG  standards  while  evaluating  not  just  interoperability  but  compatibility 
of  systems. 

6.4.2  The  Role  of  extensible  Mark-up  Language  (XML)  in  Architecture 
Documents 

During  the  course  of  this  research,  possible  investigation  into  the  belief  that  one  could 
measure  commonality  and  interoperability  in  architectures  by  looking  at  the  data  models 
of  the  architecture.  One  possible  solution  to  assure  interoperability  of  the  data  models  is 
to  use  a  common  language  like  XML  to  fonnat  the  metadata  in  the  data  dictionary  and 
data  models.  The  Department  of  Defense  has  created  an  XML  registry  to  ensure 
interoperability.  During  the  Net-Centric  Program  Assessment,  programs  were  asked 
about  their  use  of  metadata  and  conformance  to  the  DoD  Discovery  Metadata 
Specification  along  with  registration  in  the  registry.  This  registry  provides  a  baseline  set 
of  XML  Infonnation  Resources  developed  through  coordination  and  approval  among  the 
DoD  communities.  This  registry  can  be  found  on  the  World  Wide  Web  at 
http://diides.ncr.disa.mil/xmlreg/user/index.cfm.  It  would  prove  beneficial  in  using  XML 
to  develop  the  data  dictionary  in  architectures  and  provide  for  interoperability. 

6.4.3  Determining  the  Right  Number  of  Architecture  Views 

While  the  DoDAF  directs  a  specified  set  of  views  to  have  a  complete  architecture, 

during  the  course  of  this  project,  investigation  found  that  a  set  number  or  type  of  views  is 
not  an  indication  of  completeness.  Sometimes  it  is  possible  to  have  all  of  the  infonnation 
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needed  for  the  purpose  on  a  subset  of  the  required  charts,  while  other  times  more  detail  is 
required.  Recently,  the  DoDAF  version  2  was  released  and  attempts  to  start  addressing 
this  issue  with  overlays.  One  possible  research  topic  for  further  study  is  to  analyze  the 
existing  products  and  capture  the  required  infonnation  to  be  present  for  the  various 
milestone  decisions. 

6.4.4  Comparing  Architecture  Design  Methodology  -  Object  Orientated 
versus  Structured  Analysis 

In  reviewing  the  three  architectures,  it  is  interesting  to  note  that  two  were  built  with 
the  traditional  Structured  Analysis  (SA)  approach  of  systems  engineering  that  centers  on 
functional  allocation.  The  third  architecture,  CCIC2S,  was  built  using  an  object 
orientated  (00)  approach  based  on  a  software  development  model.  The  two  approaches 
are  both  covered  in  the  DoDAF,  but  which  way  is  better  or  more  able  to  produce  a 
complete  architecture  is  open  for  future  research.  A  potential  sponsor  for  a  project  like 
this  is  AFSPC/DRF  as  they  posed  similar  topics  during  discussions  with  them. 

6.4.5  Sensitivity  analysis  for  EA  Score  Card  Results 

One  critique  of  the  assessment  methods  was  having  a  single  individual  perfonn  the 

EA  Score  Card  assessment.  While  all  members  participated  in  the  analysis  of  results,  it 
was  decided  early  on  to  have  a  single  person  generate  the  scores  to  ensure  each 
architecture  so  the  scored  was  based  on  similar  interpretations  of  the  tool.  The 
conducting  of  the  score  generation  by  multiple  users  may  provide  some  insights.  The 
manpower  to  generate  the  EA  Score  Card  assessment  data  consumed  about  120  man 
hours  for  the  3  Enterprise  Architectures. 
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8  Vita 


8.1  Major  Jamison 

Major  Theresa  Jamison  graduated  from  the  University  of  Southern  Mississippi  with  a 
B.S.  in  Mathematics  and  was  commissioned  a  second  lieutenant  in  the  United  States  Air 
Force  in  May  of  1991.  Her  first  assignment  was  to  Kirtland  AFB,  New  Mexico  and  the 
Phillips  Laboratory  as  a  satellite  development  engineer.  Following  tours  to  Tyndall  AFB, 
Florida  doing  statistical  analysis  on  air-to-air  missiles;  the  Air  Force  Academy,  Colorado 
as  an  Air  Officer  Commanding;  and  Los  Angeles  AFB,  California  as  the  Director  of 
Business  Operations  for  Space-Based  Radar,  Major  Jamison  was  selected  to  attend  AFIT 
under  the  relatively  new  Intennediate  Developmental  Education  (IDE)  program. 
Following  graduation,  Major  Jamison  will  stay  at  Wright-Patterson  AFB  where  she  will 
be  in  charge  of  Weapons  and  Sensors  for  the  Air  Force’s  B-l  Bomber. 


8.2  Major  Layman 

Major  Phillip  A.  Layman  graduated  from  the  University  of  Cincinnati  with  a  B.S.  in 
Aerospace  Engineering  and  was  commissioned  a  second  lieutenant  in  the  United  States 
Air  Force  in  June  1991.  While  serving  as  a  Communications-Systems  Program  Manager, 
he  volunteered  to  cross-train  as  a  Space  and  Missile  Officer  in  1994.  Following  training, 
Major  Layman  reported  to  Minot  Air  Force  Base,  North  Dakota  where  he  held  positions 
as  a  Missile  Combat  Crew  Commander,  Flight  Commander  and  Operations  Group 
Executive  Officer.  Major  Layman  next  served  as  Missile  Warning  Crew  Commander 
and  Chief  of  Standardization  and  Evaluation  at  Thule  Air  Base,  Greenland.  After  his 
remote  tour  of  duty,  he  was  selected  as  the  Deputy  Director,  2 1st  Space  Wing  Operations 
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Center  at  Peterson  Air  Force  Base,  Colorado.  From  there,  Major  Layman  joined  the  Air 
Force  Space  Command  Inspector  General  Team.  During  that  assignment,  he  was 
selected  to  attend  Intennediate  Developmental  Education  in  AFIT’s  System  Engineering 
program.  After  graduation.  Major  Layman  is  assigned  to  United  States  Strategic 
Command  at  Offutt  Air  Force  Base  as  a  Joint  Staff  Missile  Plans  Officer. 

8.3  Major  Niska 

Major  Brice  Niska  graduated  from  the  University  of  Minnesota  with  a  B.S.  in 
Aeronautical  Engineering  and  Mechanics  and  was  commissioned  an  Air  Force  officer  in 
May  of  1991.  His  first  assignment  was  to  the  4950th  Test  Wing  as  a  Computer  Aided 
Design  Engineer.  Following  tours  to  Kirtland  AFB,  New  Mexico  as  a  Project  Manager; 
Montgomery,  Alabama  as  a  Squadron  Officer  School  Instructor;  and  Los  Angeles  AFB, 
California  as  a  Program  Manager  for  a  classified  program,  Brice  was  selected  to  attend 
AFIT  under  the  relatively  new  Intermediate  Developmental  Education  (IDE)  program. 
Following  graduation,  Major  Niska  will  head  to  Rosslyn,  Virginia  to  be  a  program 
element  monitor  for  space  at  the  Under  Secretary  of  the  Air  Force’s  Office  for  Space 
Acquisition. 


8.4  Major  Whitney 

Major  Steven  P.  Whitney  graduated  from  the  University  of  Minnesota  with  a  B.S.  in 
Electrical  Engineering  and  was  commissioned  a  second  lieutenant  in  the  United  States 
Air  Force  in  December  1992.  Major  Whitney’s  first  assignment  was  to  Schriever  AFB, 
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Colorado,  where  he  served  as  the  Chief  of  Spacecraft  Engineering  for  the  Defense 
Support  Program  satellites  performing  missile  warning  for  the  US.  Following  an 
assignment  to  the  Space  Based  Infrared  Systems  Program  Office,  Steve  was  selected  for 
the  Air  Force  Intern  Program  in  1998.  As  part  of  this,  he  served  tours  in  the  Secretary  of 
the  Air  Force’s  staff  in  Acquisitions  coordinating  budgets  and  policy  for  space  programs 
and  in  the  Office  of  the  Secretary  of  Defense  writing  Department  policy  on  Military 
Funeral  Honors,  while  completing  a  M.A.  in  Administrative  Sciences  and  Leadership 
Theory  from  the  George  Washington  University.  Just  prior  to  attending  AFIT,  Major 
Whitney  served  as  the  Director  of  Engineering  at  the  Air  Force  Communications  Facility 
at  White  Sands  Missile  Range,  New  Mexico.  Upon  graduation  from  AFIT,  Major 
Whitney  will  report  to  the  Under  Secretary  of  the  Air  Force’s  Office  for  Space 
Acquisition  as  a  program  element  monitor  for  Military  SATCOM. 


66 


Appendix  A:  EA  Score  Card  Results 


Table  12:  C2C  EA  Score  Card  Results 


Command  and  Control  Constellation 
(C2C) 

Enterprise  Architecture  Score  Card 

Clear  =  Well  defined  and  documented 

Partially  Clear  =  partially  addressed  and  documented 

Unclear  =  NOT  identified  or  addressed,  NOT  defined  or  NOT  documented 

ASC 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2  Level  of 

Partially  Clear  =  1  Alignment/ 

Unclear  =0  Integration 

Total  Status 

2 

1 

0 

Questions  to  the  enterprise 
architecture  result 

Business 

Information 

Information 

Systems 

Technology 

Infrastructure 

Factor  0-2; 

0=lnsufficient 

l=Average 

2=Full 

1 

Are  the  Mission,  Vision,  Goals, 

&  Objectives  of  the  Enterprise 
Architecture? 

2 

2 

1 

1 

0 

2 

2 

0 

2 

Is  the  Scope  of  the  enterprise 
architecture  program? 

2 

H 

1 

2 

1 

3 

1 

0 

3 

Is  the  Form  &  Function  Level 
of  deliverables? 

1 

1 

2 

1 

0 

1 

3 

0 

4 

Is  the  Business  &  IT  Strategy? 

1 

0 

0 

0 

0 

0 

1 

3 

5 

Are  the  Guiding  Principles  & 
Drivers? 

1 

1 

2 

2 

1 

2 

2 

0 

6 

Are  the  Key  Performance 
Indicators? 

0 

0 

0 

1 

0 

0 

1 

3 

7 

Are  the  Critical  Success 

Factors? 

0 

0 

1 

1 

0 

0 

2 

2 

8 

Are  the  Critical  Stakeholders? 

1 

2 

2 

1 

1 

2 

2 

0 

Sub  Score  Contextual  Level 

8 

8 

9 

9 

9 

Are  the  Collaborative  Parties 
Involved? 

1 

2 

1 

1 

1 

1 

3 

0 

10 

Are  the  Contractual 

Agreements? 

0 

0 

0 

0 

0 

0 

0 

4 

11 

Are  the  Interoperability 
standards? 

2 

2 

2 

2 

2 

4 

0 

0 

12 

Are  the  related  Law  & 
Regulations? 

2 

2 

2 

2 

2 

4 

0 

0 

13 

Is  the  Ownership  of 

Information? 

2 

2 

2 

2 

2 

4 

0 

0 

Sub  Score  Environmental  Level 

7 

8 

7 

7 

14 

Are  the  Functional 

Requirements? 

2 

2 

2 

2 

2 

4 

0 

0 

15 

Are  the  Non-Functional 
Requirements? 

2 

2 

2 

2 

2 

4 

0 

0 

16 

Are  the  concepts  in  use? 

1 

0 

0 

0 

0 

0 

1 

3 

17 

Are  the  Security  Requirements? 

1 

1 

1 

1 

1 

0 

4 

0 

18 

Are  the  Governance 
Requirements? 

1 

1 

1 

1 

1 

0 

4 

0 

Sub  Score  Conceptual  Level 

7 

6 

6 

6 

19 

Are  the  deliverables  at  logical 
level? 

1 

m 

2 

2 

1 

3 

1 

0 

20 

Are  the  critical  logical  design 
decisions? 

2 

i 

1 

1 

1 

1 

3 

0 

21 

Are  the  critical  logical  design 
decisions  traceable? 

1 

0 

0 

0 

0 

0 

1 

3 

22 

Are  the  Logical  Description 
Methods  &  Techniques? 

1 

0 

0 

0 

0 

0 

1 

3 

23 

Is  at  logical  level  the  use  of 
Modeling  Tools? 

1 

0 

0 

0 

0 

0 

1 

3 

24 

Are  the  Logical  Standards? 

0 

2 

2 

2 

1 

3 

0 

1 

Sub  Score  Logical  Level 

6 

5 

5 

5 

25 

Are  the  deliverables  at  physical 
level? 

1 

0 

0 

0 

0 

0 

1 

3 

26 

Are  the  critical  physical  design 
decision? 

0 

0 

0 

0 

0 

0 

0 

4 

67 


27 

Are  the  critical  physical  design 
decisions  traceable? 

0 

0 

0 

0 

0 

0 

0 

4 

28 

Are  the  Physical  Description 
Methods  &  Techniques? 

0 

0 

0 

0 

0 

0 

0 

4 

29 

Is  at  the  physical  level  the  use 
of  Modeling  tools 

0 

0 

0 

0 

0 

0 

0 

4 

30 

Are  the  Physical  Standards? 

0 

2 

2 

2 

1 

1 

0 

1 

Sub  Score  Physical  Level 

1 

2 

2 

2 

31 

Critical  Design  Decisions 

2 

0 

0 

1 

0 

1 

1 

2 

32 

Is  the  Organization  Impact? 

1 

1 

1 

1 

0 

0 

4 

0 

33 

Are  the  Costs  Consequences? 

1 

0 

0 

0 

1 

0 

1 

3 

34 

Is  the  Security  Impact? 

0 

0 

0 

0 

0 

0 

0 

4 

35 

Is  the  Governance  Impact? 

1 

0 

0 

0 

0 

0 

1 

3 

Sub  Score  Transformational  Level 

5 

1 

1 

2 

Total  Score  All  Levels 

34 

30 

30 

31 

Table  13:  CCIC2S  EA  Score  Card 


Combatant  Commanders  Integrated 
Command  and  Control  System  (CCIC2S) 

Enterprise  Architecture  Score  Card 

Clear  =  Well  defined  and  documented 

Partially  Clear  =  partially  addressed  and  documented 

Unclear  =  NOT  identified  or  addressed,  NOT  defined  or  NOT  documented 

ASC 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2  Level  of 

Partially  Clear  =  1  Alignment/ 

Unclear  =0  Integration 

Total  Status 

2 

1 

0 

Questions  to  the  enterprise 
architecture  result 

Business 

Information 

Information 

Systems 

Technology 

Infrastructure 

Factor  0-2; 

0=lnsufficient 

l=Average 

2=Full 

1 

Are  the  Mission,  Vision,  Goals, 

&  Objectives  of  the  Enterprise 
Architecture? 

2 

2 

2 

2 

1 

4 

0 

0 

2 

Is  the  Scope  of  the  enterprise 
architecture  program? 

2 

2 

2 

2 

2 

4 

0 

0 

3 

Is  the  Form  &  Function  Level 
of  deliverables? 

0 

2 

2 

2 

0 

3 

0 

l 

4 

Is  the  Business  &  IT  Strategy? 

1 

2 

2 

1 

1 

2 

2 

0 

5 

Are  the  Guiding  Principles  & 
Drivers? 

1 

2 

2 

2 

1 

3 

1 

0 

6 

Are  the  Key  Performance 
Indicators? 

0 

2 

2 

0 

2 

2 

0 

2 

7 

Are  the  Critical  Success 

Factors? 

1 

2 

2 

2 

2 

3 

1 

0 

8 

Are  the  Critical  Stakeholders? 

2 

2 

2 

1 

2 

3 

1 

0 

Sub  Score  Contextual  Level 

9 

16 

16 

12 

9 

Are  the  Collaborative  Parties 
Involved? 

0 

0 

0 

1 

0 

0 

1 

3 

10 

Are  the  Contractual 

Agreements? 

1 

0 

0 

1 

0 

0 

2 

2 

11 

Are  the  Interoperability 
standards? 

1 

2 

2 

2 

1 

3 

1 

0 

12 

Are  the  related  Law  & 
Regulations? 

2 

2 

2 

2 

2 

4 

0 

0 

13 

Is  the  Ownership  of 

Information? 

0 

2 

1 

1 

1 

1 

2 

1 

Sub  Score  Environmental  Level 

4 

6 

5 

7 

14 

Are  the  Functional 

Requirements? 

2 

2 

2 

2 

2 

4 

0 

0 

15 

Are  the  Non-Functional 
Requirements? 

2 

2 

2 

2 

2 

4 

0 

0 

16 

Are  the  concepts  in  use? 

1 

2 

2 

1 

2 

2 

2 

0 

17 

Are  the  Security  Requirements?  |  U 

1 

1 

1 

1 

0 

3 

0 

18 

Are  the  Governance  1 

Requirements?  |  •  ■ 

0 

1 

0 

0 

0 

1 

3 

Sub  Score  Conceptual  Level 

5 

7 

8 

6 
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19 

Are  the  deliverables  at  logical 
level? 

0 

0 

0 

0 

0 

0 

0 

4 

20 

Are  the  critical  logical  design 
decisions? 

l 

ini 

0 

0 

1 

1 

1 

2 

21 

Are  the  critical  logical  design 
decisions  traceable? 

0 

1 

0 

0 

0 

0 

1 

3 

22 

Are  the  Logical  Description 
Methods  &  Techniques? 

l 

2 

2 

2 

2 

3 

1 

0 

23 

Is  at  logical  level  the  use  of 
Modeling  Tools? 

0 

0 

0 

0 

0 

0 

0 

4 

24 

Are  the  Logical  Standards? 

1 

2 

2 

2 

2 

3 

1 

0 

<Sz/6  Score  Logical  Level 

3 

7 

4 

4 

25 

Are  the  deliverables  at  physical 
level? 

0 

0 

0 

0 

0 

0 

0 

4 

26 

Are  the  critical  physical  design 
decision? 

0 

0 

0 

0 

0 

0 

0 

4 

27 

Are  the  critical  physical  design 
decisions  traceable? 

0 

0 

0 

0 

0 

0 

0 

4 

28 

Are  the  Physical  Description 
Methods  &  Techniques? 

0 

0 

0 

0 

0 

0 

0 

4 

29 

Is  at  the  physical  level  the  use 
of  Modeling  tools 

0 

0 

0 

0 

0 

0 

0 

4 

30 

Are  the  Physical  Standards? 

0 

1 

1 

0 

1 

0 

2 

2 

■Sz/6  Score  Physical  Level 

0 

1 

1 

0 

31 

Critical  Design  Decisions 

0 

0 

0 

0 

0 

0 

0 

4 

32 

Is  the  Organization  Impact? 

1 

1 

1 

1 

1 

0 

4 

0 

33 

Are  the  Costs  Consequences? 

1 

0 

0 

0 

0 

0 

1 

3 

34 

Is  the  Security  Impact? 

0 

0 

0 

0 

0 

0 

0 

4 

35 

Is  the  Governance  Impact? 

0 

0 

0 

0 

0 

0 

0 

4 

Sub  Score  Transformational  Level 

2 

1 

1 

1 

Total  Score  All  Levels 

23 

38 

35 

30 

Table  14:  GIG  EA  Score  Card  Results 


Global  Information  Grid  (GIG) 

Enterprise  Architecture  Score  Card 

Clear  =  Well  defined  and  documented 

Partially  Clear  =  partially  addressed  and  documented 

ASC 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2 

Partially  Clear  =  1 

Unclear  =0 

Status  definition: 

Clear  =  2  Level  of 

Partially  Clear  =  1  Alignment/ 

Unclear  =0  Integration 

Total  Status 

2 

1 

0 

Questions  to  the  enterprise 
architecture  result 

Business 

Technology 

Information  1 |  Systems  |  Infrastructure 

Factor  0-2; 

0=lnsufficient 

1=  Average 

2=Full 

1 

Are  the  Mission,  Vision,  Goals, 

&  Objectives  of  the  Enterprise 
Architecture? 

2 

2 

2 

1 

1 

3 

1 

0 

2 

Is  the  Scope  of  the  enterprise 
architecture  program? 

1 

2 

2 

2 

2 

3 

1 

0 

3 

Is  the  Form  &  Function  Level 
of  deliverables? 

2 

1 

0 

0 

0 

1 

1 

2 

4 

Is  the  Business  &  IT  Strategy? 

2 

2 

2 

1 

1 

3 

1 

0 

5 

Are  the  Guiding  Principles  & 
Drivers? 

1 

1 

1 

1 

2 

0 

4 

0 

6 

Are  the  Key  Performance 
Indicators? 

2 

2 

2 

2 

4 

0 

0 

7 

Are  the  Critical  Success 

Factors? 

1 

1 

1 

1 

2 

0 

4 

0 

8 

Are  the  Critical  Stakeholders? 

2 

2 

2 

2 

2 

4 

0 

0 

Sub  Score  Contextual  Level 

13 

13 

12 

10 

9 

Are  the  Collaborative  Parties 
Involved? 

2 

2 

2 

2 

2 

4 

0 

0 

69 


10 

Are  the  Contractual 

Agreements? 

l 

0 

0 

0 

0 

0 

1 

3 

11 

Are  the  Interoperability 
standards? 

l 

1 

1 

1 

2 

0 

4 

0 

12 

Are  the  related  Law  & 
Regulations? 

l 

1 

1 

1 

2 

0 

4 

0 

13 

Is  the  Ownership  of 

Information? 

0 

1 

2 

0 

0 

1 

1 

2 

Sub  Score  Environmental  Level 

5 

5 

6 

4 

14 

Are  the  Functional 

Requirements? 

1 

1 

1 

1 

2 

0 

4 

0 

15 

Are  the  Non-Functional 
Requirements? 

1 

2 

2 

1 

1 

2 

2 

0 

16 

Are  the  concepts  in  use? 

1 

2 

2 

1 

1 

2 

2 

0 

17 

Are  the  Security  Requirements? 

2 

2 

2 

2 

2 

4 

0 

0 

18 

Are  the  Governance 
Requirements? 

2 

1 

1 

1 

1 

1 

3 

0 

Sub  Score  Conceptual  Level 

7 

8 

8 

6 

19 

Are  the  deliverables  at  logical 
level? 

1 

1 

1 

1 

2 

0 

4 

0 

20 

Are  the  critical  logical  design 
decisions? 

2 

1 

1 

1 

1 

1 

3 

0 

21 

Are  the  critical  logical  design 
decisions  traceable? 

2 

2 

2 

2 

2 

4 

0 

0 

22 

Are  the  Logical  Description 
Methods  &  Techniques? 

1 

1 

2 

2 

1 

2 

2 

0 

23 

Is  at  logical  level  the  use  of 
Modeling  Tools? 

2 

2 

2 

2 

2 

4 

0 

0 

24 

Are  the  Logical  Standards? 

1 

1 

1 

1 

2 

0 

4 

0 

Sub  Score  Logical  Level 

9 

8 

9 

9 

25 

Are  the  deliverables  at  physical 
level? 

0 

0 

1 

1 

0 

0 

2 

2 

26 

Are  the  critical  physical  design 
decision? 

0 

0 

0 

0 

0 

0 

0 

4 

27 

Are  the  critical  physical  design 
decisions  traceable? 

0 

0 

0 

0 

0 

0 

0 

4 

28 

Are  the  Physical  Description 
Methods  &  Techniques? 

1 

1 

1 

1 

1 

0 

4 

0 

29 

Is  at  the  physical  level  the  use 
of  Modeling  tools 

1 

2 

2 

2 

1 

3 

1 

0 

30 

Are  the  Physical  Standards? 

1 

2 

2 

2 

0 

3 

1 

0 

Sub  Score  Physical  Level 

3 

5 

6 

6 

31 

Critical  Design  Decisions 

1 

1 

1 

1 

2 

0 

4 

0 

32 

Is  the  Organization  Impact? 

2 

2 

2 

2 

2 

4 

0 

0 

33 

Are  the  Costs  Consequences? 

0 

0 

0 

0 

0 

0 

0 

4 

34 

Is  the  Security  Impact? 

2 

2 

2 

2 

2 

4 

0 

0 

35 

Is  the  Governance  Impact? 

1 

1 

1 

1 

2 

0 

4 

0 

Sub  Score  Transformational  Level 

6 

6 

6 

6 

Total  Score  All  Levels 

43 

45 

47 

41 

70 
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